IDEAS home Printed from https://ideas.repec.org/a/eee/gamebe/v81y2013icp215-231.html
   My bibliography  Save this article

Nonparametric learning rules from bandit experiments: The eyes have it!

Author

Listed:
  • Hu, Yingyao
  • Kayaba, Yutaka
  • Shum, Matthew

Abstract

How do people learn? We assess, in a model-free manner, subjectsʼ belief dynamics in a two-armed bandit learning experiment. A novel feature of our approach is to supplement the choice and reward data with subjectsʼ eye movements during the experiment to pin down estimates of subjectsʼ beliefs. Estimates show that subjects are more reluctant to “update down” following unsuccessful choices, than “update up” following successful choices. The profits from following the estimated learning and decision rules are smaller (by about 25% of average earnings by subjects in this experiment) than what would be obtained from a fully-rational Bayesian learning model, but comparable to the profits from alternative non-Bayesian learning models, including reinforcement learning and a simple “win-stay” choice heuristic.

Suggested Citation

  • Hu, Yingyao & Kayaba, Yutaka & Shum, Matthew, 2013. "Nonparametric learning rules from bandit experiments: The eyes have it!," Games and Economic Behavior, Elsevier, vol. 81(C), pages 215-231.
  • Handle: RePEc:eee:gamebe:v:81:y:2013:i:c:p:215-231
    DOI: 10.1016/j.geb.2013.05.003
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0899825613000766
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Daniel T. Knoepfle & Joseph Tao-yi Wang & Colin F. Camerer, 2009. "Studying Learning in Games Using Eye-Tracking," Journal of the European Economic Association, MIT Press, vol. 7(2-3), pages 388-398, 04-05.
    2. Vincent P. Crawford & Miguel A. Costa-Gomes, 2006. "Cognition and Behavior in Two-Person Guessing Games: An Experimental Study," American Economic Review, American Economic Association, vol. 96(5), pages 1737-1768, December.
    3. Christopher Anderson, 2012. "Ambiguity aversion in multi-armed bandit problems," Theory and Decision, Springer, vol. 72(1), pages 15-33, January.
    4. Xavier Gabaix & David Laibson & Guillermo Moloche & Stephen Weinberg, 2006. "Costly Information Acquisition: Experimental Analysis of a Boundedly Rational Model," American Economic Review, American Economic Association, vol. 96(4), pages 1043-1068, September.
    5. Rothschild, Michael, 1974. "A two-armed bandit theory of market pricing," Journal of Economic Theory, Elsevier, vol. 9(2), pages 185-202, October.
    6. Tat Y. Chan & Barton H. Hamilton, 2006. "Learning, Private Information, and the Economic Evaluation of Randomized Experiments," Journal of Political Economy, University of Chicago Press, vol. 114(6), pages 997-1040, December.
    7. James J. Choi & David Laibson & Brigitte C. Madrian & Andrew Metrick, 2009. "Reinforcement Learning and Savings Behavior," Journal of Finance, American Finance Association, vol. 64(6), pages 2515-2534, December.
    8. Miller, Robert A, 1984. "Job Matching and Occupational Choice," Journal of Political Economy, University of Chicago Press, vol. 92(6), pages 1086-1120, December.
    9. Noah Gans & George Knox & Rachel Croson, 2007. "Simple Models of Discrete Choice and Their Performance in Bandit Experiments," Manufacturing & Service Operations Management, INFORMS, vol. 9(4), pages 383-408, December.
    10. Gary Charness & Dan Levin, 2005. "When Optimal Choices Feel Wrong: A Laboratory Study of Bayesian Updating, Complexity, and Affect," American Economic Review, American Economic Association, vol. 95(4), pages 1300-1309, September.
    11. Hu, Yingyao, 2008. "Identification and estimation of nonlinear models with misclassification error using instrumental variables: A general solution," Journal of Econometrics, Elsevier, vol. 144(1), pages 27-61, May.
    12. Elena Reutskaja & Rosemarie Nagel & Colin F. Camerer & Antonio Rangel, 2011. "Search Dynamics in Consumer Choice under Time Pressure: An Eye-Tracking Study," American Economic Review, American Economic Association, vol. 101(2), pages 900-926, April.
    13. Vincent P. Crawford & Nagore Iriberri, 2007. "Level-k Auctions: Can a Nonequilibrium Model of Strategic Thinking Explain the Winner's Curse and Overbidding in Private-Value Auctions?," Econometrica, Econometric Society, vol. 75(6), pages 1721-1770, November.
    14. Susumu Imai & Neelam Jain & Andrew Ching, 2009. "Bayesian Estimation of Dynamic Discrete Choice Models," Econometrica, Econometric Society, vol. 77(6), pages 1865-1899, November.
    15. Jovanovic, Boyan, 1979. "Job Matching and the Theory of Turnover," Journal of Political Economy, University of Chicago Press, vol. 87(5), pages 972-990, October.
    16. Gregory S. Crawford & Matthew Shum, 2005. "Uncertainty and Learning in Pharmaceutical Demand," Econometrica, Econometric Society, vol. 73(4), pages 1137-1173, July.
    17. Robert J. Meyer & Yong Shi, 1995. "Sequential Choice Under Ambiguity: Intuitive Solutions to the Armed-Bandit Problem," Management Science, INFORMS, vol. 41(5), pages 817-834, May.
    18. Daniel A. Ackerberg, 2003. "Advertising, learning, and consumer choice in experience good markets: an empirical examination," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 44(3), pages 1007-1040, August.
    19. Patrick Bajari & Ali Hortacsu, 2005. "Are Structural Estimates of Auction Models Reasonable? Evidence from Experimental Data," Journal of Political Economy, University of Chicago Press, vol. 113(4), pages 703-741, August.
    20. Andrew Caplin & Mark Dean, 2008. "Economic Insights from "Neuroeconomic" Data," American Economic Review, American Economic Association, vol. 98(2), pages 169-174, May.
    21. Brocas, Isabelle & Camerer, Colin & Carrillo, Juan D & Wang, Stephanie W., 2009. "Measuring attention and strategic behavior in games with private information," CEPR Discussion Papers 7529, C.E.P.R. Discussion Papers.
    22. Avi Goldfarb & Mo Xiao, 2011. "Who Thinks about the Competition? Managerial Ability and Strategic Entry in US Local Telephone Markets," American Economic Review, American Economic Association, vol. 101(7), pages 3130-3161, December.
    23. Costa-Gomes, Miguel & Crawford, Vincent P & Broseta, Bruno, 2001. "Cognition and Behavior in Normal-Form Games: An Experimental Study," Econometrica, Econometric Society, vol. 69(5), pages 1193-1235, September.
    24. Nathaniel T Wilcox, 2006. "Theories of Learning in Games and Heterogeneity Bias," Econometrica, Econometric Society, vol. 74(5), pages 1271-1292, September.
    25. Weitzman, Martin L, 1979. "Optimal Search for the Best Alternative," Econometrica, Econometric Society, vol. 47(3), pages 641-654, May.
    26. Joseph Tao-yi Wang & Michael Spezio & Colin F. Camerer, 2010. "Pinocchio's Pupil: Using Eyetracking and Pupil Dilation to Understand Truth Telling and Deception in Sender-Receiver Games," American Economic Review, American Economic Association, vol. 100(3), pages 984-1007, June.
    27. Alexander L. Brown & Colin F. Camerer & Dan Lovallo, 2012. "To Review or Not to Review? Limited Strategic Thinking at the Movie Box Office," American Economic Journal: Microeconomics, American Economic Association, vol. 4(2), pages 1-26, May.
    28. Jeffrey Banks & David Porter & Mark Olson, 1997. "An experimental analysis of the bandit problem," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 10(1), pages 55-77.
    29. Hu, Yingyao & Shum, Matthew, 2012. "Nonparametric identification of dynamic models with unobserved state variables," Journal of Econometrics, Elsevier, vol. 171(1), pages 32-44.
    30. Pakes, Ariel & McGuire, Paul, 2001. "Stochastic Algorithms, Symmetric Markov Perfect Equilibrium, and the 'Curse' of Dimensionality," Econometrica, Econometric Society, vol. 69(5), pages 1261-1281, September.
    31. Bergemann, Dirk & Hege, Ulrich, 1998. "Venture capital financing, moral hazard, and learning," Journal of Banking & Finance, Elsevier, vol. 22(6-8), pages 703-735, August.
    32. K. Carrie Armel & Aurelie Beaumel & Antonio Rangel, 2008. "Biasing simple choices by manipulating relative visual attention," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 3, pages 396-403, June.
    33. Kuhnen, Camelia M. & Knutson, Brian, 2011. "The Influence of Affect on Beliefs, Preferences, and Financial Decisions," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 46(3), pages 605-626, June.
    34. Tülin Erdem & Michael P. Keane, 1996. "Decision-Making Under Uncertainty: Capturing Dynamic Brand Choice Processes in Turbulent Consumer Goods Markets," Marketing Science, INFORMS, vol. 15(1), pages 1-20.
    35. Yaw Nyarko & Andrew Schotter, 2002. "An Experimental Study of Belief Learning Using Elicited Beliefs," Econometrica, Econometric Society, vol. 70(3), pages 971-1005, May.
    36. Andrew Caplin & Mark Dean & Paul W. Glimcher & Robb B. Rutledge, 2010. "Measuring Beliefs and Rewards: A Neuroeconomic Approach," The Quarterly Journal of Economics, Oxford University Press, vol. 125(3), pages 923-960.
    37. Johnson, Eric J. & Camerer, Colin & Sen, Sankar & Rymon, Talia, 2002. "Detecting Failures of Backward Induction: Monitoring Information Search in Sequential Bargaining," Journal of Economic Theory, Elsevier, vol. 104(1), pages 16-47, May.
    38. K. Carrie Armel & Antonio Rangel, 2008. "The Impact of Computation Time and Experience on Decision Values," American Economic Review, American Economic Association, vol. 98(2), pages 163-168, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Eric Guerci & Nobuyuki Hanaki & Naoki Watanabe, 2015. "Meaningful Learning in Weighted Voting Games: An Experiment," Working Papers halshs-01216244, HAL.
    2. Hanaki, Nobuyuki & Kirman, Alan & Pezanis-Christou, Paul, 2018. "Observational and reinforcement pattern-learning: An exploratory study," European Economic Review, Elsevier, vol. 104(C), pages 1-21.
    3. Andrew T. Ching & Tülin Erdem & Michael P. Keane, 2013. "Invited Paper ---Learning Models: An Assessment of Progress, Challenges, and New Developments," Marketing Science, INFORMS, vol. 32(6), pages 913-938, November.
    4. Eric Guerci & Nobuyuki Hanaki & Naoki Watanabe, 2017. "Meaningful learning in weighted voting games: an experiment," Theory and Decision, Springer, vol. 83(1), pages 131-153, June.
    5. An, Yonghong & Hu, Yingyao & Liu, Pengfei, 2018. "Estimating heterogeneous contributing strategies in threshold public goods provision: A structural analysis," Journal of Economic Behavior & Organization, Elsevier, vol. 152(C), pages 124-146.
    6. Dimitrije Marković & Andrea M F Reiter & Stefan J Kiebel, 2019. "Predicting change: Approximate inference under explicit representation of temporal structure in changing environments," PLOS Computational Biology, Public Library of Science, vol. 15(1), pages 1-31, January.
    7. Nobuyuki Hanaki & Alan Kirman & Paul Pezanis-Christou, 2016. "Counter Intuitive Learning: An Exploratory Study," School of Economics Working Papers 2016-12, University of Adelaide, School of Economics.
    8. Hu, Yingyao, 2017. "The econometrics of unobservables: Applications of measurement error models in empirical industrial organization and labor economics," Journal of Econometrics, Elsevier, vol. 200(2), pages 154-168.
    9. Yingyao Hu, 2015. "Microeconomic models with latent variables: applications of measurement error models in empirical industrial organization and labor economics," CeMMAP working papers CWP03/15, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    10. Oyarzun, Carlos & Sanjurjo, Adam & Nguyen, Hien, 2017. "Response functions," European Economic Review, Elsevier, vol. 98(C), pages 1-31.
    11. Douglas Norton & R. Isaac, 2012. "Experts with a conflict of interest: a source of ambiguity?," Experimental Economics, Springer;Economic Science Association, vol. 15(2), pages 260-277, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Elena Reutskaja & Rosemarie Nagel & Colin F. Camerer & Antonio Rangel, 2011. "Search Dynamics in Consumer Choice under Time Pressure: An Eye-Tracking Study," American Economic Review, American Economic Association, vol. 101(2), pages 900-926, April.
    2. Caplin, Andrew & Dean, Mark, 2011. "Search, choice, and revealed preference," Theoretical Economics, Econometric Society, vol. 6(1), January.
    3. Brocas, Isabelle & Carrillo, Juan D. & Sachdeva, Ashish, 2018. "The path to equilibrium in sequential and simultaneous games: A mousetracking study," Journal of Economic Theory, Elsevier, vol. 178(C), pages 246-274.
    4. Victor Aguirregabiria & Jihye Jeon, 2020. "Firms’ Beliefs and Learning: Models, Identification, and Empirical Evidence," Review of Industrial Organization, Springer;The Industrial Organization Society, vol. 56(2), pages 203-235, March.
    5. Andrew T. Ching & Tülin Erdem & Michael P. Keane, 2013. "Invited Paper ---Learning Models: An Assessment of Progress, Challenges, and New Developments," Marketing Science, INFORMS, vol. 32(6), pages 913-938, November.
    6. Andrew T. Ching & Tülin Erdem & Michael P. Keane, 2013. "Learning Models: An Assessment of Progress, Challenges and New Developments," Economics Papers 2013-W07, Economics Group, Nuffield College, University of Oxford.
    7. Song Lin & Juanjuan Zhang & John R. Hauser, 2015. "Learning from Experience, Simply," Marketing Science, INFORMS, vol. 34(1), pages 1-19, January.
    8. Hu, Yingyao & Shum, Matthew, 2012. "Nonparametric identification of dynamic models with unobserved state variables," Journal of Econometrics, Elsevier, vol. 171(1), pages 32-44.
    9. Georganas, Sotiris & Healy, Paul J. & Weber, Roberto A., 2015. "On the persistence of strategic sophistication," Journal of Economic Theory, Elsevier, vol. 159(PA), pages 369-400.
    10. Sorensen, Morten, 2007. "Learning by Investing: Evidence from Venture Capital," SIFR Research Report Series 53, Institute for Financial Research.
    11. Giovanna Devetag & Sibilla Guida & Luca Polonio, 2016. "An eye-tracking study of feature-based choice in one-shot games," Experimental Economics, Springer;Economic Science Association, vol. 19(1), pages 177-201, March.
    12. Taisuke Imai & Min Jeong Kang & Colin F. Camerer, 2019. "When the eyes say buy: visual fixations during hypothetical consumer choice improve prediction of actual purchases," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 5(1), pages 112-122, August.
    13. Matthew P. Taylor, 2017. "Information Acquisition Under Risky Conditions Across Real And Hypothetical Settings," Economic Inquiry, Western Economic Association International, vol. 55(1), pages 352-367, January.
    14. Nagel, Rosemarie & Bühren, Christoph & Frank, Björn, 2017. "Inspired and inspiring: Hervé Moulin and the discovery of the beauty contest game," Mathematical Social Sciences, Elsevier, vol. 90(C), pages 191-207.
    15. Guidon Fenig & Giovanni Gallipoli & Yoram Halevy, 2018. "Piercing the 'Payoff Function' Veil: Tracing Beliefs and Motives," Working Papers tecipa-619, University of Toronto, Department of Economics.
    16. Marcoul, Philippe & Weninger, Quinn, 2008. "Search and active learning with correlated information: Empirical evidence from mid-Atlantic clam fishermen," Journal of Economic Dynamics and Control, Elsevier, vol. 32(6), pages 1921-1948, June.
    17. Xavier Gabaix, 2017. "Behavioral Inattention," NBER Working Papers 24096, National Bureau of Economic Research, Inc.
    18. Tore Ellingsen & Robert Östling, 2010. "When Does Communication Improve Coordination?," American Economic Review, American Economic Association, vol. 100(4), pages 1695-1724, September.
    19. Brocas, Isabelle & Camerer, Colin & Carrillo, Juan D & Wang, Stephanie W., 2009. "Measuring attention and strategic behavior in games with private information," CEPR Discussion Papers 7529, C.E.P.R. Discussion Papers.
    20. Polonio, Luca & Di Guida, Sibilla & Coricelli, Giorgio, 2015. "Strategic sophistication and attention in games: An eye-tracking study," Games and Economic Behavior, Elsevier, vol. 94(C), pages 80-96.

    More about this item

    Keywords

    Learning; Belief dynamics; Experiments; Eye tracking; Bayesian vs. non-Bayesian learning;
    All these keywords.

    JEL classification:

    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:gamebe:v:81:y:2013:i:c:p:215-231. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Nithya Sathishkumar). General contact details of provider: http://www.elsevier.com/locate/inca/622836 .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.