IDEAS home Printed from https://ideas.repec.org/a/eee/gamebe/v70y2010i1p71-83.html
   My bibliography  Save this article

A payoff-based learning procedure and its application to traffic games

Author

Listed:
  • Cominetti, Roberto
  • Melo, Emerson
  • Sorin, Sylvain

Abstract

A stochastic process that describes a payoff-based learning procedure and the associated adaptive behavior of players in a repeated game is considered. The process is shown to converge almost surely towards a stationary state which is characterized as an equilibrium for a related game. The analysis is based on techniques borrowed from the theory of stochastic algorithms and proceeds by studying an associated continuous dynamical system which represents the evolution of the players' evaluations. An application to the case of finitely many users in a congested traffic network with parallel links is considered. Alternative descriptions for the dynamics and the corresponding rest points are discussed, including a Lagrangian representation.

Suggested Citation

  • Cominetti, Roberto & Melo, Emerson & Sorin, Sylvain, 2010. "A payoff-based learning procedure and its application to traffic games," Games and Economic Behavior, Elsevier, vol. 70(1), pages 71-83, September.
  • Handle: RePEc:eee:gamebe:v:70:y:2010:i:1:p:71-83
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0899-8256(08)00220-0
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Cascetta, Ennio, 1989. "A stochastic process approach to the analysis of temporal dynamics in transportation networks," Transportation Research Part B: Methodological, Elsevier, vol. 23(1), pages 1-17, February.
    2. Michel Benaïm & Josef Hofbauer & Sylvain Sorin, 2006. "Stochastic Approximations and Differential Inclusions, Part II: Applications," Mathematics of Operations Research, INFORMS, vol. 31(4), pages 673-695, November.
    3. Erel Avineri & Joseph Prashker, 2006. "The Impact of Travel Time Information on Travelers’ Learning under Uncertainty," Transportation, Springer, vol. 33(4), pages 393-408, July.
    4. Sandholm, William H., 2001. "Potential Games with Continuous Player Sets," Journal of Economic Theory, Elsevier, vol. 97(1), pages 81-108, March.
    5. Erev, Ido & Roth, Alvin E, 1998. "Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria," American Economic Review, American Economic Association, vol. 88(4), pages 848-881, September.
    6. Laslier, Jean-Francois & Topol, Richard & Walliser, Bernard, 2001. "A Behavioral Learning Process in Games," Games and Economic Behavior, Elsevier, vol. 37(2), pages 340-366, November.
    7. Arthur, W Brian, 1993. "On Designing Economic Agents That Behave Like Human Agents," Journal of Evolutionary Economics, Springer, vol. 3(1), pages 1-22, February.
    8. Sergiu Hart, 2013. "Adaptive Heuristics," World Scientific Book Chapters, in: Simple Adaptive Strategies From Regret-Matching to Uncoupled Dynamics, chapter 11, pages 253-287, World Scientific Publishing Co. Pte. Ltd..
    9. Foster, Dean P. & Vohra, Rakesh V., 1997. "Calibrated Learning and Correlated Equilibrium," Games and Economic Behavior, Elsevier, vol. 21(1-2), pages 40-55, October.
    10. Beggs, A.W., 2005. "On the convergence of reinforcement learning," Journal of Economic Theory, Elsevier, vol. 122(1), pages 1-36, May.
    11. William H. Sandholm, 2002. "Evolutionary Implementation and Congestion Pricing," Review of Economic Studies, Oxford University Press, vol. 69(3), pages 667-689.
    12. Michel Benaim & Josef Hofbauer & Sylvain Sorin, 2005. "Stochastic Approximations and Differential Inclusions II: Applications," Levine's Bibliography 784828000000000098, UCLA Department of Economics.
    13. Martin Posch, 1997. "Cycling in a stochastic learning algorithm for normal form games," Journal of Evolutionary Economics, Springer, vol. 7(2), pages 193-207.
    14. Borgers, Tilman & Sarin, Rajiv, 1997. "Learning Through Reinforcement and Replicator Dynamics," Journal of Economic Theory, Elsevier, vol. 77(1), pages 1-14, November.
    15. McKelvey Richard D. & Palfrey Thomas R., 1995. "Quantal Response Equilibria for Normal Form Games," Games and Economic Behavior, Elsevier, vol. 10(1), pages 6-38, July.
    16. Josef Hofbauer & William H. Sandholm, 2002. "On the Global Convergence of Stochastic Fictitious Play," Econometrica, Econometric Society, vol. 70(6), pages 2265-2294, November.
    17. Young, H. Peyton, 2004. "Strategic Learning and its Limits," OUP Catalogue, Oxford University Press, number 9780199269181.
    18. Duffy, John & Hopkins, Ed, 2005. "Learning, information, and sorting in market entry games: theory and evidence," Games and Economic Behavior, Elsevier, vol. 51(1), pages 31-62, April.
    19. Selten, R. & Chmura, T. & Pitz, T. & Kube, S. & Schreckenberg, M., 2007. "Commuters route choice behaviour," Games and Economic Behavior, Elsevier, vol. 58(2), pages 394-406, February.
    20. Benaim, Michel & Hirsch, Morris W., 1999. "Mixed Equilibria and Dynamical Systems Arising from Fictitious Play in Perturbed Games," Games and Economic Behavior, Elsevier, vol. 29(1-2), pages 36-72, October.
    21. Drew Fudenberg & David K. Levine, 1998. "The Theory of Learning in Games," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262061945.
    22. Freund, Yoav & Schapire, Robert E., 1999. "Adaptive Game Playing Using Multiplicative Weights," Games and Economic Behavior, Elsevier, vol. 29(1-2), pages 79-103, October.
    23. Horowitz, Joel L., 1984. "The stability of stochastic equilibrium in a two-link transportation network," Transportation Research Part B: Methodological, Elsevier, vol. 18(1), pages 13-28, February.
    24. Monderer, Dov & Shapley, Lloyd S., 1996. "Potential Games," Games and Economic Behavior, Elsevier, vol. 14(1), pages 124-143, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bravo, Mario & Mertikopoulos, Panayotis, 2017. "On the robustness of learning in games with stochastically perturbed payoff observations," Games and Economic Behavior, Elsevier, vol. 103(C), pages 41-66.
    2. Erhao Xie, 2019. "Monetary Payoff and Utility Function in Adaptive Learning Models," Staff Working Papers 19-50, Bank of Canada.
    3. Pierre Coucheney & Bruno Gaujal & Panayotis Mertikopoulos, 2015. "Penalty-Regulated Dynamics and Robust Learning Procedures in Games," Mathematics of Operations Research, INFORMS, vol. 40(3), pages 611-633, March.
    4. Naoki Funai, 2019. "Convergence results on stochastic adaptive learning," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 68(4), pages 907-934, November.
    5. Roy Allen & John Rehbeck, 2021. "A Generalization of Quantal Response Equilibrium via Perturbed Utility," Games, MDPI, Open Access Journal, vol. 12(1), pages 1-16, March.
    6. Farokhi, Farhad & Johansson, Karl H., 2015. "A piecewise-constant congestion taxing policy for repeated routing games," Transportation Research Part B: Methodological, Elsevier, vol. 78(C), pages 123-143.
    7. Khai Xiang Chiong & Alfred Galichon & Matt Shum, 2016. "Duality in dynamic discrete‐choice models," Quantitative Economics, Econometric Society, vol. 7(1), pages 83-115, March.
    8. Khai Xiang Chiong & Alfred Galichon & Matt Shum, 2021. "Duality in dynamic discrete-choice models," Papers 2102.06076, arXiv.org, revised Feb 2021.
    9. Manxi Wu & Saurabh Amin & Asuman Ozdaglar, 2021. "Multi-agent Bayesian Learning with Best Response Dynamics: Convergence and Stability," Papers 2109.00719, arXiv.org.
    10. Manxi Wu & Saurabh Amin, 2019. "Securing Infrastructure Facilities: When Does Proactive Defense Help?," Dynamic Games and Applications, Springer, vol. 9(4), pages 984-1025, December.
    11. Mario Bravo, 2016. "An Adjusted Payoff-Based Procedure for Normal Form Games," Mathematics of Operations Research, INFORMS, vol. 41(4), pages 1469-1483, November.
    12. Panayotis Mertikopoulos & William H. Sandholm, 2016. "Learning in Games via Reinforcement and Regularization," Mathematics of Operations Research, INFORMS, vol. 41(4), pages 1297-1324, November.
    13. Alfredo Garcia & Mingyi Hong & Jorge Barrera, 2012. "“Cap and Trade” for Congestion Control," Dynamic Games and Applications, Springer, vol. 2(3), pages 280-293, September.
    14. Boyer, Sebastien & Blandin, Sebastien & Wynter, Laura, 2015. "Stability of transportation networks under adaptive routing policies," Transportation Research Part B: Methodological, Elsevier, vol. 81(P3), pages 886-903.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Panayotis Mertikopoulos & William H. Sandholm, 2016. "Learning in Games via Reinforcement and Regularization," Mathematics of Operations Research, INFORMS, vol. 41(4), pages 1297-1324, November.
    2. Benaïm, Michel & Hofbauer, Josef & Hopkins, Ed, 2009. "Learning in games with unstable equilibria," Journal of Economic Theory, Elsevier, vol. 144(4), pages 1694-1709, July.
    3. Cason, Timothy N. & Friedman, Daniel & Hopkins, Ed, 2010. "Testing the TASP: An experimental investigation of learning in games with unstable equilibria," Journal of Economic Theory, Elsevier, vol. 145(6), pages 2309-2331, November.
    4. Kets, W., 2007. "The Minority Game : An Economics Perspective," Other publications TiSEM 65d52a6a-b27d-45a9-93a7-e, Tilburg University, School of Economics and Management.
    5. Hopkins, Ed & Posch, Martin, 2005. "Attainability of boundary points under reinforcement learning," Games and Economic Behavior, Elsevier, vol. 53(1), pages 110-125, October.
    6. Hofbauer, Josef & Hopkins, Ed, 2005. "Learning in perturbed asymmetric games," Games and Economic Behavior, Elsevier, vol. 52(1), pages 133-152, July.
    7. Leslie, David S. & Collins, E.J., 2006. "Generalised weakened fictitious play," Games and Economic Behavior, Elsevier, vol. 56(2), pages 285-298, August.
    8. Michel Benaïm & Josef Hofbauer & Sylvain Sorin, 2006. "Stochastic Approximations and Differential Inclusions, Part II: Applications," Mathematics of Operations Research, INFORMS, vol. 31(4), pages 673-695, November.
    9. Sandholm, William H., 2015. "Population Games and Deterministic Evolutionary Dynamics," Handbook of Game Theory with Economic Applications,, Elsevier.
    10. Dridi, Slimane & Lehmann, Laurent, 2014. "On learning dynamics underlying the evolution of learning rules," Theoretical Population Biology, Elsevier, vol. 91(C), pages 20-36.
    11. Ianni, Antonella, 2014. "Learning strict Nash equilibria through reinforcement," Journal of Mathematical Economics, Elsevier, vol. 50(C), pages 148-155.
    12. Kets, W., 2008. "Networks and learning in game theory," Other publications TiSEM 7713fce1-3131-498c-8c6f-3, Tilburg University, School of Economics and Management.
    13. Beggs, A.W., 2005. "On the convergence of reinforcement learning," Journal of Economic Theory, Elsevier, vol. 122(1), pages 1-36, May.
    14. William Sandholm, 2014. "Probabilistic Interpretations of Integrability for Game Dynamics," Dynamic Games and Applications, Springer, vol. 4(1), pages 95-106, March.
    15. Jacques Durieu & Philippe Solal, 2012. "Models of Adaptive Learning in Game Theory," Chapters, in: Richard Arena & Agnès Festré & Nathalie Lazaric (ed.), Handbook of Knowledge and Economics, chapter 11, Edward Elgar Publishing.
    16. Chernov, G. & Susin, I., 2019. "Models of learning in games: An overview," Journal of the New Economic Association, New Economic Association, vol. 44(4), pages 77-125.
    17. Mario Bravo, 2016. "An Adjusted Payoff-Based Procedure for Normal Form Games," Mathematics of Operations Research, INFORMS, vol. 41(4), pages 1469-1483, November.
    18. Candogan, Ozan & Ozdaglar, Asuman & Parrilo, Pablo A., 2013. "Dynamics in near-potential games," Games and Economic Behavior, Elsevier, vol. 82(C), pages 66-90.
    19. Georgios Chasparis & Jeff Shamma & Anders Rantzer, 2015. "Nonconvergence to saddle boundary points under perturbed reinforcement learning," International Journal of Game Theory, Springer;Game Theory Society, vol. 44(3), pages 667-699, August.
    20. Ed Hopkins, 2002. "Two Competing Models of How People Learn in Games," Econometrica, Econometric Society, vol. 70(6), pages 2141-2166, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:gamebe:v:70:y:2010:i:1:p:71-83. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.elsevier.com/locate/inca/622836 .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/inca/622836 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.