IDEAS home Printed from https://ideas.repec.org/a/eee/eecrev/v104y2018icp1-21.html
   My bibliography  Save this article

Observational and reinforcement pattern-learning: An exploratory study

Author

Listed:
  • Hanaki, Nobuyuki
  • Kirman, Alan
  • Pezanis-Christou, Paul

Abstract

Understanding how individuals learn in an unknown environment is an important problem in economics. We model and examine experimentally behavior in a very simple multi-armed bandit framework in which participants do not know the inter-temporal payoff structure. We propose a baseline reinforcement learning model that allows for pattern-recognition and change in the strategy space. We also analyse three augmented versions that accommodate observational learning from the actions and/or payoffs of another player. The models successfully reproduce the distributional properties of observed discovery times and total payoffs. Our study further shows that when one of the pair discovers the hidden pattern, observing another’s actions and/or payoffs improves discovery time compared to the baseline case.

Suggested Citation

  • Hanaki, Nobuyuki & Kirman, Alan & Pezanis-Christou, Paul, 2018. "Observational and reinforcement pattern-learning: An exploratory study," European Economic Review, Elsevier, vol. 104(C), pages 1-21.
    • Handle: RePEc:eee:eecrev:v:104:y:2018:i:c:p:1-21
    DOI: 10.1016/j.euroecorev.2018.01.009
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0014292118300187
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.euroecorev.2018.01.009?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Erev, Ido & Roth, Alvin E, 1998. "Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria," American Economic Review, American Economic Association, vol. 88(4), pages 848-881, September.
    2. James G. March, 1991. "Exploration and Exploitation in Organizational Learning," Organization Science, INFORMS, vol. 2(1), pages 71-87, February.
    3. Ralph-C. Bayer & Hang Wu, 2013. "Do We Learn from Our Own Experience or from Observing Others?," School of Economics and Public Policy Working Papers 2013-21, University of Adelaide, School of Economics and Public Policy.
    4. Sonsino, Doron, 1997. "Learning to Learn, Pattern Recognition, and Nash Equilibrium," Games and Economic Behavior, Elsevier, vol. 18(2), pages 286-331, February.
    5. Ben Greiner, 2015. "Subject pool recruitment procedures: organizing experiments with ORSEE," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 1(1), pages 114-125, July.
    6. Bossan, Benjamin & Jann, Ole & Hammerstein, Peter, 2015. "The evolution of social learning and its economic consequences," Journal of Economic Behavior & Organization, Elsevier, vol. 112(C), pages 266-288.
    7. Spiliopoulos, Leonidas, 2012. "Pattern recognition and subjective belief learning in a repeated constant-sum game," Games and Economic Behavior, Elsevier, vol. 75(2), pages 921-935.
    8. Jeffrey Banks & David Porter & Mark Olson, 1997. "An experimental analysis of the bandit problem," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 10(1), pages 55-77.
    9. Bray, Margaret, 1982. "Learning, estimation, and the stability of rational expectations," Journal of Economic Theory, Elsevier, vol. 26(2), pages 318-339, April.
    10. Spiliopoulos, Leonidas, 2013. "Beyond fictitious play beliefs: Incorporating pattern recognition and similarity matching," Games and Economic Behavior, Elsevier, vol. 81(C), pages 69-85.
    11. Herrnstein, R J, 1991. "Experiments on Stable Suboptimality in Individual Behavior," American Economic Review, American Economic Association, vol. 81(2), pages 360-364, May.
    12. McKinney, C. Nicholas & Van Huyck, John B., 2013. "Eureka Learning: Heuristics and response time in perfect information games," Games and Economic Behavior, Elsevier, vol. 79(C), pages 223-232.
    13. Charles A. Holt & Susan K. Laury, 2002. "Risk Aversion and Incentive Effects," American Economic Review, American Economic Association, vol. 92(5), pages 1644-1655, December.
    14. Colin Camerer & Teck-Hua Ho, 1999. "Experience-weighted Attraction Learning in Normal Form Games," Econometrica, Econometric Society, vol. 67(4), pages 827-874, July.
    15. Woodford, Michael, 1990. "Learning to Believe in Sunspots," Econometrica, Econometric Society, vol. 58(2), pages 277-307, March.
    16. Hu, Yingyao & Kayaba, Yutaka & Shum, Matthew, 2013. "Nonparametric learning rules from bandit experiments: The eyes have it!," Games and Economic Behavior, Elsevier, vol. 81(C), pages 215-231.
    17. Urs Fischbacher, 2007. "z-Tree: Zurich toolbox for ready-made economic experiments," Experimental Economics, Springer;Economic Science Association, vol. 10(2), pages 171-178, June.
    18. Lones Smith & Peter Norman Sørensen, 2011. "observational learning," The New Palgrave Dictionary of Economics,, Palgrave Macmillan.
    19. Armantier, Olivier, 2004. "Does observation influence learning?," Games and Economic Behavior, Elsevier, vol. 46(2), pages 221-239, February.
    20. Dufwenberg, Martin & Sundaram, Ramya & Butler, David J., 2010. "Epiphany in the Game of 21," Journal of Economic Behavior & Organization, Elsevier, vol. 75(2), pages 132-143, August.
    21. Daniella Laureiro-Martínez & Stefano Brusoni & Nicola Canessa & Maurizio Zollo, 2015. "Understanding the exploration–exploitation dilemma: An fMRI study of attention control and decision-making performance," Strategic Management Journal, Wiley Blackwell, vol. 36(3), pages 319-338, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jean Paul Rabanal & Aleksei Chernulich & John Horowitz & Olga A. Rud & Manizha Sharifova, 2019. "Market timing under public and private information," Working Papers 151, Peruvian Economic Association.
    2. Eric Innocenti & Corinne Idda & Dominique Prunetti & Pierre-Régis Gonsolin, 2022. "Agent-based modelling of a small-scale fishery in Corsica," Post-Print hal-03886619, HAL.
    3. Nobuyuki Hanaki & Ali I. Ozkes, 2023. "Strategic environment effect and communication," Experimental Economics, Springer;Economic Science Association, vol. 26(3), pages 588-621, July.
    4. Chernulich, Aleksei & Horowitz, John & Rabanal, Jean Paul & Rud, Olga A & Sharifova , Manizha, 2021. "Entry and exit decisions under public and private information: An experiment," UiS Working Papers in Economics and Finance 2021/3, University of Stavanger.
    5. Aleksei Chernulich & John Horowitz & Jean Paul Rabanal & Olga Rud & Manizha Sharifova, 2023. "Entry and exit decisions under public and private information: an experiment," Experimental Economics, Springer;Economic Science Association, vol. 26(2), pages 339-356, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nobuyuki Hanaki & Alan P. Kirman & Paul Pezanis-Christou, 2016. "Counter Intuitive Learning: An Exploratory Study," CESifo Working Paper Series 6029, CESifo.
    2. Eric Guerci & Nobuyuki Hanaki & Naoki Watanabe, 2017. "Meaningful learning in weighted voting games: an experiment," Theory and Decision, Springer, vol. 83(1), pages 131-153, June.
    3. Masiliūnas, Aidas, 2023. "Learning in rent-seeking contests with payoff risk and foregone payoff information," Games and Economic Behavior, Elsevier, vol. 140(C), pages 50-72.
    4. Eric Guerci & Nobuyuki Hanaki & Naoki Watanabe, 2015. "Meaningful Learning in Weighted Voting Games: An Experiment," Working Papers halshs-01216244, HAL.
    5. Ioannou, Christos A. & Romero, Julian, 2014. "A generalized approach to belief learning in repeated games," Games and Economic Behavior, Elsevier, vol. 87(C), pages 178-203.
    6. Naoki Watanabe, 2022. "Reconsidering Meaningful Learning in a Bandit Experiment on Weighted Voting: Subjects’ Search Behavior," The Review of Socionetwork Strategies, Springer, vol. 16(1), pages 81-107, April.
    7. Dennis A. V. Dittrich & Werner Güth & Martin G. Kocher & Paul Pezanis‐Christou, 2012. "Loss Aversion and Learning to Bid," Economica, London School of Economics and Political Science, vol. 79(314), pages 226-257, April.
    8. Oyarzun, Carlos & Sanjurjo, Adam & Nguyen, Hien, 2017. "Response functions," European Economic Review, Elsevier, vol. 98(C), pages 1-31.
    9. Shafran, Aric P., 2012. "Learning in games with risky payoffs," Games and Economic Behavior, Elsevier, vol. 75(1), pages 354-371.
    10. Wen, Yuanji, 2018. "Voluntary information acquisition in an asymmetric-Information game:comparing learning theories in the laboratory," Journal of Economic Behavior & Organization, Elsevier, vol. 150(C), pages 202-219.
    11. Masiliūnas, Aidas, 2017. "Overcoming coordination failure in a critical mass game: Strategic motives and action disclosure," Journal of Economic Behavior & Organization, Elsevier, vol. 139(C), pages 214-251.
    12. Hommes, Cars, 2018. "Behavioral & experimental macroeconomics and policy analysis: a complex systems approach," Working Paper Series 2201, European Central Bank.
    13. Alina Ferecatu & Arnaud De Bruyn, 2022. "Understanding Managers’ Trade-Offs Between Exploration and Exploitation," Marketing Science, INFORMS, vol. 41(1), pages 139-165, January.
    14. Galliera, Arianna, 2018. "Self-selecting random or cumulative pay? A bargaining experiment," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 72(C), pages 106-120.
    15. Embrey, Matthew & Hyndman, Kyle & Riedl, Arno, 2021. "Bargaining with a residual claimant: An experimental study," Games and Economic Behavior, Elsevier, vol. 126(C), pages 335-354.
    16. Andreas Nicklisch, 2011. "Learning strategic environments: an experimental study of strategy formation and transfer," Theory and Decision, Springer, vol. 71(4), pages 539-558, October.
    17. Boosey, Luke & Brookins, Philip & Ryvkin, Dmitry, 2017. "Contests with group size uncertainty: Experimental evidence," Games and Economic Behavior, Elsevier, vol. 105(C), pages 212-229.
    18. Lorko, Matej & Servátka, Maroš & Zhang, Le, 2023. "Hidden inefficiency: Strategic inflation of project schedules," Journal of Economic Behavior & Organization, Elsevier, vol. 206(C), pages 313-326.
    19. Elmaghraby, Wedad J. & Larson, Nathan, 2012. "Explaining deviations from equilibrium in auctions with avoidable fixed costs," Games and Economic Behavior, Elsevier, vol. 76(1), pages 131-159.
    20. Rick, Scott & Weber, Roberto A., 2010. "Meaningful learning and transfer of learning in games played repeatedly without feedback," Games and Economic Behavior, Elsevier, vol. 68(2), pages 716-730, March.

    More about this item

    Keywords

    Multi-armed bandit; Reinforcement learning; Payoff patterns; Observational learning;
    All these keywords.

    JEL classification:

    • D81 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Criteria for Decision-Making under Risk and Uncertainty
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:eecrev:v:104:y:2018:i:c:p:1-21. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eer .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.