IDEAS home Printed from https://ideas.repec.org/a/aea/aecrev/v96y2006i4p1029-1042.html
   My bibliography  Save this article

The Speed of Learning in Noisy Games: Partial Reinforcement and the Sustainability of Cooperation

Author

Listed:
  • Yoella Bereby-Meyer
  • Alvin E. Roth

Abstract

In an experiment, players? ability to learn to cooperate in the repeated prisoner?s dilemma was substantially diminished when the payoffs were noisy, even though players could monitor one another?s past actions perfectly. In contrast, in one-time play against a succession of opponents, noisy payoffs increased cooperation, by slowing the rate at which cooperation decays. These observations are consistent with the robust observation from the psychology literature that partial reinforcement (adding randomness to the link between an action and its consequences while holding expected payoffs constant) slows learning. This effect is magnified in the repeated game: when others are slow to learn to cooperate, the benefits of cooperation are reduced, which further hampers cooperation. These results show that a small change in the payoff environment, which changes the speed of individual learning, can have a large effect on collective behavior. And they show that there may be interesting comparative dynamics that can be derived from careful attention to the fact that at least some economic behavior is learned from experience. (JEL C71, C72, C73, D83)

Suggested Citation

  • Yoella Bereby-Meyer & Alvin E. Roth, 2006. "The Speed of Learning in Noisy Games: Partial Reinforcement and the Sustainability of Cooperation," American Economic Review, American Economic Association, vol. 96(4), pages 1029-1042, September.
  • Handle: RePEc:aea:aecrev:v:96:y:2006:i:4:p:1029-1042
    Note: DOI: 10.1257/aer.96.4.1029
    as

    Download full text from publisher

    File URL: http://www.aeaweb.org/articles.php?doi=10.1257/aer.96.4.1029
    Download Restriction: no

    File URL: http://www.aeaweb.org/aer/data/sept06/20030187_data.zip
    Download Restriction: Access to full text is restricted to AEA members and institutional subscribers.
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Selten, Reinhard & Stoecker, Rolf, 1986. "End behavior in sequences of finite Prisoner's Dilemma supergames A learning theory approach," Journal of Economic Behavior & Organization, Elsevier, vol. 7(1), pages 47-70, March.
    2. Roth, Alvin E. & Erev, Ido, 1995. "Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term," Games and Economic Behavior, Elsevier, vol. 8(1), pages 164-212.
    3. Roth, Alvin E. & Sonmez, Tayfun & Utku Unver, M., 2005. "Pairwise kidney exchange," Journal of Economic Theory, Elsevier, vol. 125(2), pages 151-188, December.
    4. Atila Abdulkadiroğlu & Parag A. Pathak & Alvin E. Roth, 2005. "The New York City High School Match," American Economic Review, American Economic Association, vol. 95(2), pages 364-367, May.
    5. Kreps, David M. & Milgrom, Paul & Roberts, John & Wilson, Robert, 1982. "Rational cooperation in the finitely repeated prisoners' dilemma," Journal of Economic Theory, Elsevier, vol. 27(2), pages 245-252, August.
    6. Nick Feltovich & John Duffy, 1999. "Does observation of others affect learning in strategic environments? An experimental study," International Journal of Game Theory, Springer;Game Theory Society, vol. 28(1), pages 131-152.
    7. Green, Edward J & Porter, Robert H, 1984. "Noncooperative Collusion under Imperfect Price Information," Econometrica, Econometric Society, vol. 52(1), pages 87-100, January.
    8. Alvin E. Roth & Tayfun Sönmez, 2005. "A Kidney Exchange Clearinghouse in New England," American Economic Review, American Economic Association, vol. 95(2), pages 376-380, May.
    9. Alvin E. Roth & Tayfun Sönmez & M. Utku Ünver, 2004. "Kidney Exchange," The Quarterly Journal of Economics, Oxford University Press, vol. 119(2), pages 457-488.
    10. Sainty, Barbara, 1999. "Achieving greater cooperation in a noisy prisoner's dilemma: an experimental investigation," Journal of Economic Behavior & Organization, Elsevier, vol. 39(4), pages 421-435, July.
    11. Milgrom,Paul, 2004. "Putting Auction Theory to Work," Cambridge Books, Cambridge University Press, number 9780521536721, October.
    12. Cooper, Russell & DeJong, Douglas V. & Forsythe, Robert & Ross, Thomas W., 1996. "Cooperation without Reputation: Experimental Evidence from Prisoner's Dilemma Games," Games and Economic Behavior, Elsevier, vol. 12(2), pages 187-218, February.
    13. Stahl, Dale O., 2000. "Rule Learning in Symmetric Normal-Form Games: Theory and Evidence," Games and Economic Behavior, Elsevier, vol. 32(1), pages 105-138, July.
    14. Erev, Ido & Roth, Alvin E, 1998. "Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria," American Economic Review, American Economic Association, vol. 88(4), pages 848-881, September.
    15. Miller, John H., 1996. "The coevolution of automata in the repeated Prisoner's Dilemma," Journal of Economic Behavior & Organization, Elsevier, vol. 29(1), pages 87-112, January.
    16. Atila Abdulkadiroğlu & Parag A. Pathak & Alvin E. Roth & Tayfun Sönmez, 2005. "The Boston Public School Match," American Economic Review, American Economic Association, vol. 95(2), pages 368-371, May.
    17. Esther Hauk & Rosemarie Nagel, 2000. "Choice of partners in multiple two-person prisoner's dilemma games: An experimental study," Economics Working Papers 487, Department of Economics and Business, Universitat Pompeu Fabra.
    18. Erev, Ido & Bereby-Meyer, Yoella & Roth, Alvin E., 1999. "The effect of adding a constant to all payoffs: experimental investigation, and implications for reinforcement learning models," Journal of Economic Behavior & Organization, Elsevier, vol. 39(1), pages 111-128, May.
    19. Colin Camerer & Teck-Hua Ho, 1999. "Experience-weighted Attraction Learning in Normal Form Games," Econometrica, Econometric Society, vol. 67(4), pages 827-874, July.
    20. Andreoni, James A & Miller, John H, 1993. "Rational Cooperation in the Finitely Repeated Prisoner's Dilemma: Experimental Evidence," Economic Journal, Royal Economic Society, vol. 103(418), pages 570-585, May.
    21. Dale O. Stahl, 1999. "Evidence based rules and learning in symmetric normal-form games," International Journal of Game Theory, Springer;Game Theory Society, vol. 28(1), pages 111-130.
    22. Kahneman, Daniel & Tversky, Amos, 1979. "Prospect Theory: An Analysis of Decision under Risk," Econometrica, Econometric Society, vol. 47(2), pages 263-291, March.
    23. Per Molander, 1985. "The Optimal Level of Generosity in a Selfish, Uncertain Environment," Journal of Conflict Resolution, Peace Science Society (International), vol. 29(4), pages 611-618, December.
    24. Drew Fudenberg & David K. Levine, 1998. "The Theory of Learning in Games," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262061945.
    25. Nick Feltovich, 2000. "Reinforcement-Based vs. Belief-Based Learning Models in Experimental Asymmetric-Information," Econometrica, Econometric Society, vol. 68(3), pages 605-642, May.
    26. Jonathan Bendor, 1993. "Uncertainty and the Evolution of Cooperation," Journal of Conflict Resolution, Peace Science Society (International), vol. 37(4), pages 709-734, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Todd Guilfoos & Andreas Pape, 2016. "Predicting human cooperation in the Prisoner’s Dilemma using case-based decision theory," Theory and Decision, Springer, vol. 80(1), pages 1-32, January.
    2. Grosskopf, Brit & Roth, Alvin E., 2009. "If you are offered the Right of First Refusal, should you accept? An investigation of contract design," Games and Economic Behavior, Elsevier, vol. 65(1), pages 176-204, January.
    3. Duffy, John, 2006. "Agent-Based Models and Human Subject Experiments," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 19, pages 949-1011, Elsevier.
    4. Committee, Nobel Prize, 2012. "Alvin E. Roth and Lloyd S. Shapley: Stable allocations and the practice of market design," Nobel Prize in Economics documents 2012-1, Nobel Prize Committee.
    5. Scott Duke Kominers & Alexander Teytelboym & Vincent P Crawford, 2017. "An invitation to market design," Oxford Review of Economic Policy, Oxford University Press, vol. 33(4), pages 541-571.
    6. Asim Ansari & Ricardo Montoya & Oded Netzer, 2012. "Dynamic learning in behavioral games: A hidden Markov mixture of experts approach," Quantitative Marketing and Economics (QME), Springer, vol. 10(4), pages 475-503, December.
    7. Hanaki, Nobuyuki & Sethi, Rajiv & Erev, Ido & Peterhansl, Alexander, 2005. "Learning strategies," Journal of Economic Behavior & Organization, Elsevier, vol. 56(4), pages 523-542, April.
    8. Haruvy, Ernan & Stahl, Dale O., 2012. "Between-game rule learning in dissimilar symmetric normal-form games," Games and Economic Behavior, Elsevier, vol. 74(1), pages 208-221.
    9. Camerer, Colin F. & Ho, Teck-Hua & Chong, Juin-Kuan, 2002. "Sophisticated Experience-Weighted Attraction Learning and Strategic Teaching in Repeated Games," Journal of Economic Theory, Elsevier, vol. 104(1), pages 137-188, May.
    10. Rapoport, Amnon & Stein, William E. & Parco, James E. & Nicholas, Thomas E., 2003. "Equilibrium play and adaptive learning in a three-person centipede game," Games and Economic Behavior, Elsevier, vol. 43(2), pages 239-265, May.
    11. Alvin E Roth & Tayfun Sönmez & M. Utku Ünver, 2005. "Efficient Kidney Exchange: Coincidence of Wants in a Structured Market," Levine's Bibliography 784828000000000126, UCLA Department of Economics.
    12. Alvin Roth, 2008. "Deferred acceptance algorithms: history, theory, practice, and open questions," International Journal of Game Theory, Springer;Game Theory Society, vol. 36(3), pages 537-569, March.
    13. Alvin E. Roth, 2009. "What Have We Learned from Market Design?," Innovation Policy and the Economy, University of Chicago Press, vol. 9(1), pages 79-112.
    14. Gunnthorsdottir, Anna & Rapoport, Amnon, 2006. "Embedding social dilemmas in intergroup competition reduces free-riding," Organizational Behavior and Human Decision Processes, Elsevier, vol. 101(2), pages 184-199, November.
    15. Ho, Teck H. & Camerer, Colin F. & Chong, Juin-Kuan, 2007. "Self-tuning experience weighted attraction learning in games," Journal of Economic Theory, Elsevier, vol. 133(1), pages 177-198, March.
    16. Camerer, Colin F. & Ho, Teck-Hua, 2015. "Behavioral Game Theory Experiments and Modeling," Handbook of Game Theory with Economic Applications,, Elsevier.
    17. Alvin E. Roth, 2008. "What Have We Learned from Market Design?," Economic Journal, Royal Economic Society, vol. 118(527), pages 285-310, March.
    18. Mitropoulos, Atanasios, 2001. "Learning under minimal information: An experiment on mutual fate control," Journal of Economic Psychology, Elsevier, vol. 22(4), pages 523-557, August.
    19. Gary Bolton, 1998. "Bargaining and Dilemma Games: From Laboratory Data Towards Theoretical Synthesis," Experimental Economics, Springer;Economic Science Association, vol. 1(3), pages 257-281, December.
    20. Haruvy, Ernan & Roth, Alvin E. & Unver, M. Utku, 2006. "The dynamics of law clerk matching: An experimental and computational investigation of proposals for reform of the market," Journal of Economic Dynamics and Control, Elsevier, vol. 30(3), pages 457-486, March.

    More about this item

    JEL classification:

    • C71 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory - - - Cooperative Games
    • C72 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory - - - Noncooperative Games
    • C73 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory - - - Stochastic and Dynamic Games; Evolutionary Games
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aea:aecrev:v:96:y:2006:i:4:p:1029-1042. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/aeaaaea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Michael P. Albert (email available below). General contact details of provider: https://edirc.repec.org/data/aeaaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.