IDEAS home Printed from https://ideas.repec.org/p/vir/virpap/327.html
   My bibliography  Save this paper

Stochastic Game Theory: Adjustment to Equilibrium Under Noisy Directional Learning

Author

Listed:
  • Simon P. Anderson
  • Jacob K. Goeree
  • Charles A. Holt

Abstract

This paper presents a dynamic model in which agents adjust their decisions in the direction of higher payoffs, subject to random error. This process produces a probability distribution of players' decisions whose evolution over time is determined by the Fokker-Planck equation. The dynamic process is stable for all potential games, a class of payoff structures that includes several widely studied games. In equilibrium, the distributions that determine expected payoffs correspond to the distributions that arise from the logit function applied to those expected payoffs. This "logit equilibrium" forms a stochastic generalization of the Nash equilibrium and provides a possible explanation of anomalous laboratory data.

Suggested Citation

  • Simon P. Anderson & Jacob K. Goeree & Charles A. Holt, 1999. "Stochastic Game Theory: Adjustment to Equilibrium Under Noisy Directional Learning," Virginia Economics Online Papers 327, University of Virginia, Department of Economics.
    • Handle: RePEc:vir:virpap:327
    as

    Download full text from publisher

    File URL: http://repec.as.virginia.edu/RePEc/vir/virpap/papers/virpap327.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. A. Roth & I. Er’ev, 2010. "Learning in Extensive Form Games: Experimental Data and Simple Dynamic Models in the Intermediate Run," Levine's Working Paper Archive 387, David K. Levine.
    2. McKelvey Richard D. & Palfrey Thomas R., 1995. "Quantal Response Equilibria for Normal Form Games," Games and Economic Behavior, Elsevier, vol. 10(1), pages 6-38, July.
    3. Young, H Peyton, 1993. "The Evolution of Conventions," Econometrica, Econometric Society, vol. 61(1), pages 57-84, January.
    4. Roth, Alvin E. & Erev, Ido, 1995. "Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term," Games and Economic Behavior, Elsevier, vol. 8(1), pages 164-212.
    5. Fudenberg, D. & Harris, C., 1992. "Evolutionary dynamics with aggregate shocks," Journal of Economic Theory, Elsevier, vol. 57(2), pages 420-441, August.
    6. Kandori, Michihiro & Mailath, George J & Rob, Rafael, 1993. "Learning, Mutation, and Long Run Equilibria in Games," Econometrica, Econometric Society, vol. 61(1), pages 29-56, January.
    7. Fernando Vega-Redondo, 1997. "The Evolution of Walrasian Behavior," Econometrica, Econometric Society, vol. 65(2), pages 375-384, March.
    8. Simon P. Anderson & Jacob K. Goeree & Charles A. Holt, 1998. "Rent Seeking with Bounded Rationality: An Analysis of the All-Pay Auction," Journal of Political Economy, University of Chicago Press, vol. 106(4), pages 828-853, August.
    9. Binmore, K. & Samuelson, L., 1995. "Evolutionary Drift and Equilibrium Selection," Working papers 9529, Wisconsin Madison - Social Systems.
    10. C. Monica Capra, 1999. "Anomalous Behavior in a Traveler's Dilemma?," American Economic Review, American Economic Association, vol. 89(3), pages 678-690, June.
    11. ANDERSON, Simon P. & de PALMA, André & THISSE, Jacques-François, 1992. "Interpretations of the logit discrete choice models and the theory of product differentiation," LIDAM Reprints CORE 1017, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    12. Rhode, Paul & Stegeman, Mark, 2001. "Non-Nash equilibria of Darwinian dynamics with applications to duopoly," International Journal of Industrial Organization, Elsevier, vol. 19(3-4), pages 415-453, March.
    13. Goeree, Jacob K. & Holt, Charles A., 2005. "An experimental study of costly coordination," Games and Economic Behavior, Elsevier, vol. 51(2), pages 349-364, May.
    14. Slade, Margaret E, 1994. "What Does an Oligopoly Maximize?," Journal of Industrial Economics, Wiley Blackwell, vol. 42(1), pages 45-61, March.
    15. Anderson, Simon P. & Goeree, Jacob K. & Holt, Charles A., 1998. "A theoretical analysis of altruism and decision error in public goods games," Journal of Public Economics, Elsevier, vol. 70(2), pages 297-323, November.
    16. Monderer, Dov & Shapley, Lloyd S., 1996. "Potential Games," Games and Economic Behavior, Elsevier, vol. 14(1), pages 124-143, May.
    17. Rosenthal, Robert W, 1989. "A Bounded-Rationality Approach to the Study of Noncooperative Games," International Journal of Game Theory, Springer;Game Theory Society, vol. 18(3), pages 273-291.
    18. Smith, Vernon L & Walker, James M, 1993. "Monetary Rewards and Decision Cost in Experimental Economics," Economic Inquiry, Western Economic Association International, vol. 31(2), pages 245-261, April.
    19. Erev, Ido & Roth, Alvin E, 1998. "Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria," American Economic Review, American Economic Association, vol. 88(4), pages 848-881, September.
    20. Chen, Hsiao-Chi & Friedman, James W. & Thisse, Jacques-Francois, 1997. "Boundedly Rational Nash Equilibrium: A Probabilistic Choice Approach," Games and Economic Behavior, Elsevier, vol. 18(1), pages 32-54, January.
    21. Ken Binmore & Larry Samuelson, 1999. "Evolutionary Drift and Equilibrium Selection," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 66(2), pages 363-393.
    22. Binmore Kenneth G. & Samuelson Larry & Vaughan Richard, 1995. "Musical Chairs: Modeling Noisy Evolution," Games and Economic Behavior, Elsevier, vol. 11(1), pages 1-35, October.
    23. John Bryant, 1983. "A Simple Rational Expectations Keynes-type Model," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 98(3), pages 525-528.
    24. Ken Binmore & Larry Samuelson, "undated". "Evolutionary Drift And Equilibrium Selection," ELSE working papers 049, ESRC Centre on Economics Learning and Social Evolution.
    25. Binmore, K. & Samuelson, L. & Vaughan, R., 1993. "Musical Chairs: Modelling Noisy Evolution," Working papers 9324, Wisconsin Madison - Social Systems.
    26. Selten, Reinhard & Joachim Buchta, 1994. "Experimental Sealed Bid First Price Auctions with Directly Observed Bid Functions," Discussion Paper Serie B 270, University of Bonn, Germany.
    27. Ken Binmore & Larry Samuelson, "undated". "Evolutionary Drift and Equilibrium Selection," ELSE working papers 011, ESRC Centre on Economics Learning and Social Evolution.
    28. Ochs Jack, 1995. "Games with Unique, Mixed Strategy Equilibria: An Experimental Study," Games and Economic Behavior, Elsevier, vol. 10(1), pages 202-217, July.
    29. Binmore, Ken & Samuelson, Larry, 1997. "Muddling Through: Noisy Equilibrium Selection," Journal of Economic Theory, Elsevier, vol. 74(2), pages 235-265, June.
    30. C. Monica Capra & Charles A. Holt, 1999. "Coordination," Southern Economic Journal, John Wiley & Sons, vol. 65(3), pages 630-636, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. Article Summary: Noisy Directional Learning and the Logit Equilibrium
      by John Barrdear in John Barrdear on 2009-08-24 18:38:19

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. C. Monica Capra & Jacob K Goeree & Rosario Gomez & Charles A Holt, 2002. "Learning and Noisy Equilibrium Behavior in an Experimental Study of Imperfect Price Competition," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 43(3), pages 613-636, August.
    2. Veronika Grimm & Jaromir Kovarik & Giovanni Ponti, 2008. "Fixed price plus rationing: an experiment," Experimental Economics, Springer;Economic Science Association, vol. 11(4), pages 402-422, December.
    3. Basov, S., 2001. "An Evolutionary Model of Reciprocity," Department of Economics - Working Papers Series 812, The University of Melbourne.
    4. Jiabin Wu, 2021. "Stochastic Value Formation," Dynamic Games and Applications, Springer, vol. 11(3), pages 597-611, September.
    5. Charles A. Holt & Jacob K. Goeree, 1999. "Stochastic Game Theory: For Playing Games, Not Just for Doing Theory," Virginia Economics Online Papers 306, University of Virginia, Department of Economics.
    6. Philip A. Haile & Ali Hortaçsu & Grigory Kosenok, 2008. "On the Empirical Content of Quantal Response Equilibrium," American Economic Review, American Economic Association, vol. 98(1), pages 180-200, March.
    7. Suren Basov, 2005. "Bounded rationality: static versus dynamic approaches," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 25(4), pages 871-885, June.
    8. Michihiro Kandori, 2003. "The Erosion and Sustainability of Norms and Morale," The Japanese Economic Review, Japanese Economic Association, vol. 54(1), pages 29-48, March.
    9. Hofbauer,J. & Sandholm,W.H., 2001. "Evolution and learning in games with randomly disturbed payoffs," Working papers 5, Wisconsin Madison - Social Systems.
    10. Goeree, Jacob K. & Holt, Charles A., 2005. "An experimental study of costly coordination," Games and Economic Behavior, Elsevier, vol. 51(2), pages 349-364, May.
    11. Simon P Anderson & Jacob K Goeree & Charles A Holt, 2001. "A Thoeretical Anlysis of Altruism and Decision Error in Public Goods Games," Levine's Working Paper Archive 563824000000000075, David K. Levine.
    12. Basov, S., 2001. "A Noisy Model of Individual Behaviour," Department of Economics - Working Papers Series 791, The University of Melbourne.
    13. JIMENEZ Edward, 2002. "Unified Game Theory," Computing in Economics and Finance 2002 25, Society for Computational Economics.
    14. C. Monica Capra & Susana Cabrera & Rosario Gómez, 2003. "The Effects of Common Advice on One-shot Traveler’s Dilemma Games: Explaining Behavior through an Introspective Model with Errors," Economic Working Papers at Centro de Estudios Andaluces E2003/17, Centro de Estudios Andaluces.
    15. Simon P. Anderson & Jacob K. Goeree & Charles A. Holt, 2002. "The Logit Equilibrium: A Perspective on Intuitive Behavioral Anomalies," Southern Economic Journal, John Wiley & Sons, vol. 69(1), pages 21-47, July.
    16. Hofbauer,J. & Sandholm,W.H., 2001. "Evolution and learning in games with randomly disturbed payoffs," Working papers 5, Wisconsin Madison - Social Systems.
    17. Armantier, Olivier, 2004. "Does observation influence learning?," Games and Economic Behavior, Elsevier, vol. 46(2), pages 221-239, February.
    18. Stegeman, Mark & Rhode, Paul, 2004. "Stochastic Darwinian equilibria in small and large populations," Games and Economic Behavior, Elsevier, vol. 49(1), pages 171-214, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Simon P. Anderson & Jacob K. Goeree & Charles A. Holt, 2004. "Noisy Directional Learning and the Logit Equilibrium," Scandinavian Journal of Economics, Wiley Blackwell, vol. 106(3), pages 581-602, October.
    2. Simon P. Anderson & Jacob K. Goeree & Charles A. Holt, 2002. "The Logit Equilibrium: A Perspective on Intuitive Behavioral Anomalies," Southern Economic Journal, John Wiley & Sons, vol. 69(1), pages 21-47, July.
    3. Anderson, Simon P. & Goeree, Jacob K. & Holt, Charles A., 2001. "Minimum-Effort Coordination Games: Stochastic Potential and Logit Equilibrium," Games and Economic Behavior, Elsevier, vol. 34(2), pages 177-199, February.
    4. Ianni, A., 2002. "Reinforcement learning and the power law of practice: some analytical results," Discussion Paper Series In Economics And Econometrics 203, Economics Division, School of Social Sciences, University of Southampton.
    5. Ponti, Giovanni, 2000. "Continuous-time evolutionary dynamics: theory and practice," Research in Economics, Elsevier, vol. 54(2), pages 187-214, June.
    6. Sandholm, William H., 2007. "Pigouvian pricing and stochastic evolutionary implementation," Journal of Economic Theory, Elsevier, vol. 132(1), pages 367-382, January.
    7. Jonathan Newton, 2018. "Evolutionary Game Theory: A Renaissance," Games, MDPI, vol. 9(2), pages 1-67, May.
    8. Hofbauer,J. & Sandholm,W.H., 2001. "Evolution and learning in games with randomly disturbed payoffs," Working papers 5, Wisconsin Madison - Social Systems.
    9. Goeree, Jacob K. & Holt, Charles A., 2005. "An experimental study of costly coordination," Games and Economic Behavior, Elsevier, vol. 51(2), pages 349-364, May.
    10. Jacob K. Goeree & Charles A. Holt, 2001. "Ten Little Treasures of Game Theory and Ten Intuitive Contradictions," American Economic Review, American Economic Association, vol. 91(5), pages 1402-1422, December.
    11. Battalio,R. & Samuelson,L. & Huyck,J. van, 1998. "Risk dominance, payoff dominance and probabilistic choice learning," Working papers 2, Wisconsin Madison - Social Systems.
    12. Hofbauer,J. & Sandholm,W.H., 2001. "Evolution and learning in games with randomly disturbed payoffs," Working papers 5, Wisconsin Madison - Social Systems.
    13. Mielke, Jahel & Steudle, Gesine A., 2018. "Green Investment and Coordination Failure: An Investors' Perspective," Ecological Economics, Elsevier, vol. 150(C), pages 88-95.
    14. Hofbauer, Josef & Sandholm, William H., 2007. "Evolution in games with randomly disturbed payoffs," Journal of Economic Theory, Elsevier, vol. 132(1), pages 47-69, January.
    15. Benndorf, Volker & Martínez-Martínez, Ismael & Normann, Hans-Theo, 2021. "Games with coupled populations: An experiment in continuous time," Journal of Economic Theory, Elsevier, vol. 195(C).
    16. Sandholm,W.H., 1999. "Markov evolution with inexact information," Working papers 15, Wisconsin Madison - Social Systems.
    17. Dai, Darong, 2012. "On the Existence and Stability of Pareto Optimal Endogenous Matching with Fairness," MPRA Paper 40560, University Library of Munich, Germany.
    18. Sandholm, William H., 2003. "Evolution and equilibrium under inexact information," Games and Economic Behavior, Elsevier, vol. 44(2), pages 343-378, August.
    19. Alessandro Innocenti & Mauro Caminati & Roberto Ricciuti, 2003. "Drift effect and timing without observability: experimental evidence," Department of Economics University of Siena 405, Department of Economics, University of Siena.
    20. Ponti, Giovanni, 2000. "Cycles of Learning in the Centipede Game," Games and Economic Behavior, Elsevier, vol. 30(1), pages 115-141, January.

    More about this item

    Keywords

    bounded rationality; noisy directional learning; Fokker- Planck equation; potential games; logit equilibrium; stochastic potential.;
    All these keywords.

    JEL classification:

    • C62 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Existence and Stability Conditions of Equilibrium
    • C73 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory - - - Stochastic and Dynamic Games; Evolutionary Games

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:vir:virpap:327. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Debby Stanford (email available below). General contact details of provider: http://www.virginia.edu/economics/home.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.