IDEAS home Printed from https://ideas.repec.org/p/jhu/papers/560.html
   My bibliography  Save this paper

Nonparametric Learning Rules from Bandit Experiments: The Eyes have it!

Author

Listed:
  • Yingyao Hu
  • Yutaka Kayaba
  • Matt Shum

Abstract

How do people learn? We assess, in a distribution-free manner, subjects?learning and choice rules in dynamic two-armed bandit (probabilistic reversal learning) experiments. To aid in identification and estimation, we use auxiliary measures of subjects?beliefs, in the form of their eye-movements during the experiment. Our estimated choice probabilities and learning rules have some distinctive features; notably that subjects tend to update in a non-smooth manner following choices made in accordance with current beliefs. Moreover, the beliefs implied by our nonparametric learning rules are closer to those from a (non-Bayesian) reinforcement learning model, than a Bayesian learning model.

Suggested Citation

  • Yingyao Hu & Yutaka Kayaba & Matt Shum, 2010. "Nonparametric Learning Rules from Bandit Experiments: The Eyes have it!," Economics Working Paper Archive 560, The Johns Hopkins University,Department of Economics.
  • Handle: RePEc:jhu:papers:560
    as

    Download full text from publisher

    File URL: http://www.econ2.jhu.edu/REPEC/papers/wp560.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Elena Reutskaja & Rosemarie Nagel & Colin F. Camerer & Antonio Rangel, 2011. "Search Dynamics in Consumer Choice under Time Pressure: An Eye-Tracking Study," American Economic Review, American Economic Association, vol. 101(2), pages 900-926, April.
    2. Vincent P. Crawford & Nagore Iriberri, 2007. "Level-k Auctions: Can a Nonequilibrium Model of Strategic Thinking Explain the Winner's Curse and Overbidding in Private-Value Auctions?," Econometrica, Econometric Society, vol. 75(6), pages 1721-1770, November.
    3. Weitzman, Martin L, 1979. "Optimal Search for the Best Alternative," Econometrica, Econometric Society, vol. 47(3), pages 641-654, May.
    4. Susumu Imai & Neelam Jain & Andrew Ching, 2009. "Bayesian Estimation of Dynamic Discrete Choice Models," Econometrica, Econometric Society, vol. 77(6), pages 1865-1899, November.
    5. Daniel T. Knoepfle & Joseph Tao-yi Wang & Colin F. Camerer, 2009. "Studying Learning in Games Using Eye-Tracking," Journal of the European Economic Association, MIT Press, vol. 7(2-3), pages 388-398, 04-05.
    6. Bergemann, Dirk & Hege, Ulrich, 1998. "Venture capital financing, moral hazard, and learning," Journal of Banking & Finance, Elsevier, vol. 22(6-8), pages 703-735, August.
    7. James J. Choi & David Laibson & Brigitte C. Madrian & Andrew Metrick, 2009. "Reinforcement Learning and Savings Behavior," Journal of Finance, American Finance Association, vol. 64(6), pages 2515-2534, December.
    8. Costa-Gomes, Miguel & Crawford, Vincent P & Broseta, Bruno, 2001. "Cognition and Behavior in Normal-Form Games: An Experimental Study," Econometrica, Econometric Society, vol. 69(5), pages 1193-1235, September.
    9. Vincent P. Crawford & Miguel A. Costa-Gomes, 2006. "Cognition and Behavior in Two-Person Guessing Games: An Experimental Study," American Economic Review, American Economic Association, vol. 96(5), pages 1737-1768, December.
    10. Tülin Erdem & Michael P. Keane, 1996. "Decision-Making Under Uncertainty: Capturing Dynamic Brand Choice Processes in Turbulent Consumer Goods Markets," Marketing Science, INFORMS, vol. 15(1), pages 1-20.
    11. Jovanovic, Boyan, 1979. "Job Matching and the Theory of Turnover," Journal of Political Economy, University of Chicago Press, vol. 87(5), pages 972-990, October.
    12. Gregory S. Crawford & Matthew Shum, 2005. "Uncertainty and Learning in Pharmaceutical Demand," Econometrica, Econometric Society, vol. 73(4), pages 1137-1173, July.
    13. Johan Lauwereyns & Katsumi Watanabe & Brian Coe & Okihide Hikosaka, 2002. "A neural correlate of response bias in monkey caudate nucleus," Nature, Nature, vol. 418(6896), pages 413-417, July.
    14. Jeffrey Banks & David Porter & Mark Olson, 1997. "An experimental analysis of the bandit problem," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 10(1), pages 55-77.
    15. Yaw Nyarko & Andrew Schotter, 2002. "An Experimental Study of Belief Learning Using Elicited Beliefs," Econometrica, Econometric Society, vol. 70(3), pages 971-1005, May.
    16. Hu, Yingyao & Shum, Matthew, 2012. "Nonparametric identification of dynamic models with unobserved state variables," Journal of Econometrics, Elsevier, vol. 171(1), pages 32-44.
    17. Pakes, Ariel & McGuire, Paul, 2001. "Stochastic Algorithms, Symmetric Markov Perfect Equilibrium, and the 'Curse' of Dimensionality," Econometrica, Econometric Society, vol. 69(5), pages 1261-1281, September.
    18. Nathaniel D. Daw & John P. O'Doherty & Peter Dayan & Ben Seymour & Raymond J. Dolan, 2006. "Cortical substrates for exploratory decisions in humans," Nature, Nature, vol. 441(7095), pages 876-879, June.
    19. Christopher Anderson, 2012. "Ambiguity aversion in multi-armed bandit problems," Theory and Decision, Springer, vol. 72(1), pages 15-33, January.
    20. Robert J. Meyer & Yong Shi, 1995. "Sequential Choice Under Ambiguity: Intuitive Solutions to the Armed-Bandit Problem," Management Science, INFORMS, vol. 41(5), pages 817-834, May.
    21. Xavier Gabaix & David Laibson & Guillermo Moloche & Stephen Weinberg, 2006. "Costly Information Acquisition: Experimental Analysis of a Boundedly Rational Model," American Economic Review, American Economic Association, vol. 96(4), pages 1043-1068, September.
    22. Patrick Bajari & Ali Hortacsu, 2005. "Are Structural Estimates of Auction Models Reasonable? Evidence from Experimental Data," Journal of Political Economy, University of Chicago Press, vol. 113(4), pages 703-741, August.
    23. Rothschild, Michael, 1974. "A two-armed bandit theory of market pricing," Journal of Economic Theory, Elsevier, vol. 9(2), pages 185-202, October.
    24. Tat Y. Chan & Barton H. Hamilton, 2006. "Learning, Private Information, and the Economic Evaluation of Randomized Experiments," Journal of Political Economy, University of Chicago Press, vol. 114(6), pages 997-1040, December.
    25. Andrew Caplin & Mark Dean & Paul W. Glimcher & Robb B. Rutledge, 2010. "Measuring Beliefs and Rewards: A Neuroeconomic Approach," The Quarterly Journal of Economics, Oxford University Press, vol. 125(3), pages 923-960.
    26. Daniel A. Ackerberg, 2003. "Advertising, learning, and consumer choice in experience good markets: an empirical examination," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 44(3), pages 1007-1040, August.
    27. Miller, Robert A, 1984. "Job Matching and Occupational Choice," Journal of Political Economy, University of Chicago Press, vol. 92(6), pages 1086-1120, December.
    28. Noah Gans & George Knox & Rachel Croson, 2007. "Simple Models of Discrete Choice and Their Performance in Bandit Experiments," Manufacturing & Service Operations Management, INFORMS, vol. 9(4), pages 383-408, December.
    29. Andrew Caplin & Mark Dean, 2008. "Economic Insights from "Neuroeconomic" Data," American Economic Review, American Economic Association, vol. 98(2), pages 169-174, May.
    30. Kuhnen, Camelia M. & Knutson, Brian, 2011. "The Influence of Affect on Beliefs, Preferences, and Financial Decisions," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 46(3), pages 605-626, June.
    31. Carrillo, Juan & Camerer, Colin & Brocas, Isabelle & Wang, Stephanie W., 2009. "Measuring attention and strategic behavior in games with private information," CEPR Discussion Papers 7529, C.E.P.R. Discussion Papers.
    32. Avi Goldfarb & Mo Xiao, 2011. "Who Thinks about the Competition? Managerial Ability and Strategic Entry in US Local Telephone Markets," American Economic Review, American Economic Association, vol. 101(7), pages 3130-3161, December.
    33. Johnson, Eric J. & Camerer, Colin & Sen, Sankar & Rymon, Talia, 2002. "Detecting Failures of Backward Induction: Monitoring Information Search in Sequential Bargaining," Journal of Economic Theory, Elsevier, vol. 104(1), pages 16-47, May.
    34. Nathaniel T Wilcox, 2006. "Theories of Learning in Games and Heterogeneity Bias," Econometrica, Econometric Society, vol. 74(5), pages 1271-1292, September.
    35. K. Carrie Armel & Antonio Rangel, 2008. "The Impact of Computation Time and Experience on Decision Values," American Economic Review, American Economic Association, vol. 98(2), pages 163-168, May.
    36. James J. Choi & David Laibson & Brigitte C. Madrian & Andrew Metrick, 2009. "Reinforcement Learning and Savings Behavior," Journal of Finance, American Finance Association, vol. 64(6), pages 2515-2534, December.
    37. Gary Charness & Dan Levin, 2005. "When Optimal Choices Feel Wrong: A Laboratory Study of Bayesian Updating, Complexity, and Affect," American Economic Review, American Economic Association, vol. 95(4), pages 1300-1309, September.
    38. Elise Payzan-LeNestour & Peter Bossaerts, 2011. "Risk, Unexpected Uncertainty, and Estimation Uncertainty: Bayesian Learning in Unstable Settings," PLOS Computational Biology, Public Library of Science, vol. 7(1), pages 1-14, January.
    39. Joseph Tao-yi Wang & Michael Spezio & Colin F. Camerer, 2010. "Pinocchio's Pupil: Using Eyetracking and Pupil Dilation to Understand Truth Telling and Deception in Sender-Receiver Games," American Economic Review, American Economic Association, vol. 100(3), pages 984-1007, June.
    40. Hu, Yingyao, 2008. "Identification and estimation of nonlinear models with misclassification error using instrumental variables: A general solution," Journal of Econometrics, Elsevier, vol. 144(1), pages 27-61, May.
    41. Alexander L. Brown & Colin F. Camerer & Dan Lovallo, 2012. "To Review or Not to Review? Limited Strategic Thinking at the Movie Box Office," American Economic Journal: Microeconomics, American Economic Association, vol. 4(2), pages 1-26, May.
    42. repec:cup:judgdm:v:3:y:2008:i::p:396-403 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Eric Guerci & Nobuyuki Hanaki & Naoki Watanabe, 2015. "Meaningful Learning in Weighted Voting Games: An Experiment," Working Papers halshs-01216244, HAL.
    2. Eric Guerci & Nobuyuki Hanaki & Naoki Watanabe, 2017. "Meaningful learning in weighted voting games: an experiment," Theory and Decision, Springer, vol. 83(1), pages 131-153, June.
    3. Stephan Billinger & Kannan Srikanth & Nils Stieglitz & Terry R. Schumacher, 2021. "Exploration and exploitation in complex search tasks: How feedback influences whether and where human agents search," Strategic Management Journal, Wiley Blackwell, vol. 42(2), pages 361-385, February.
    4. Hanaki, Nobuyuki & Kirman, Alan & Pezanis-Christou, Paul, 2018. "Observational and reinforcement pattern-learning: An exploratory study," European Economic Review, Elsevier, vol. 104(C), pages 1-21.
    5. Nobuyuki Hanaki & Alan Kirman & Paul Pezanis-Christou, 2016. "Counter Intuitive Learning: An Exploratory Study," School of Economics and Public Policy Working Papers 2016-12, University of Adelaide, School of Economics and Public Policy.
    6. Douglas Norton & R. Isaac, 2012. "Experts with a conflict of interest: a source of ambiguity?," Experimental Economics, Springer;Economic Science Association, vol. 15(2), pages 260-277, June.
    7. An, Yonghong & Hu, Yingyao & Liu, Pengfei, 2018. "Estimating heterogeneous contributing strategies in threshold public goods provision: A structural analysis," Journal of Economic Behavior & Organization, Elsevier, vol. 152(C), pages 124-146.
    8. Carlos Alós-Ferrer & Alexander Jaudas & Alexander Ritschel, 2021. "Effortful Bayesian updating: A pupil-dilation study," Journal of Risk and Uncertainty, Springer, vol. 63(1), pages 81-102, August.
    9. Hu, Yingyao, 2017. "The econometrics of unobservables: Applications of measurement error models in empirical industrial organization and labor economics," Journal of Econometrics, Elsevier, vol. 200(2), pages 154-168.
    10. Naoki Watanabe, 2022. "Reconsidering Meaningful Learning in a Bandit Experiment on Weighted Voting: Subjects’ Search Behavior," The Review of Socionetwork Strategies, Springer, vol. 16(1), pages 81-107, April.
    11. Yingyao Hu, 2015. "Microeconomic models with latent variables: applications of measurement error models in empirical industrial organization and labor economics," CeMMAP working papers 03/15, Institute for Fiscal Studies.
    12. Andrew T. Ching & Tülin Erdem & Michael P. Keane, 2013. "Invited Paper ---Learning Models: An Assessment of Progress, Challenges, and New Developments," Marketing Science, INFORMS, vol. 32(6), pages 913-938, November.
    13. Dimitrije Marković & Andrea M F Reiter & Stefan J Kiebel, 2019. "Predicting change: Approximate inference under explicit representation of temporal structure in changing environments," PLOS Computational Biology, Public Library of Science, vol. 15(1), pages 1-31, January.
    14. Yingyao Hu, 2015. "Microeconomic models with latent variables: applications of measurement error models in empirical industrial organization and labor economics," CeMMAP working papers CWP03/15, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    15. Oyarzun, Carlos & Sanjurjo, Adam & Nguyen, Hien, 2017. "Response functions," European Economic Review, Elsevier, vol. 98(C), pages 1-31.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hu, Yingyao, 2017. "The Econometrics of Unobservables -- Latent Variable and Measurement Error Models and Their Applications in Empirical Industrial Organization and Labor Economics [The Econometrics of Unobservables]," Economics Working Paper Archive 64578, The Johns Hopkins University,Department of Economics, revised 2021.
    2. Elena Reutskaja & Rosemarie Nagel & Colin F. Camerer & Antonio Rangel, 2011. "Search Dynamics in Consumer Choice under Time Pressure: An Eye-Tracking Study," American Economic Review, American Economic Association, vol. 101(2), pages 900-926, April.
    3. , & ,, 2011. "Search, choice, and revealed preference," Theoretical Economics, Econometric Society, vol. 6(1), January.
    4. Brocas, Isabelle & Carrillo, Juan D. & Sachdeva, Ashish, 2018. "The path to equilibrium in sequential and simultaneous games: A mousetracking study," Journal of Economic Theory, Elsevier, vol. 178(C), pages 246-274.
    5. Song Lin & Juanjuan Zhang & John R. Hauser, 2015. "Learning from Experience, Simply," Marketing Science, INFORMS, vol. 34(1), pages 1-19, January.
    6. Victor Aguirregabiria & Jihye Jeon, 2020. "Firms’ Beliefs and Learning: Models, Identification, and Empirical Evidence," Review of Industrial Organization, Springer;The Industrial Organization Society, vol. 56(2), pages 203-235, March.
    7. Andrew T. Ching & Tülin Erdem & Michael P. Keane, 2013. "Invited Paper ---Learning Models: An Assessment of Progress, Challenges, and New Developments," Marketing Science, INFORMS, vol. 32(6), pages 913-938, November.
    8. Andrew T. Ching & Tülin Erdem & Michael P. Keane, 2013. "Learning Models: An Assessment of Progress, Challenges and New Developments," Economics Papers 2013-W07, Economics Group, Nuffield College, University of Oxford.
    9. Guidon Fenig & Giovanni Gallipoli & Yoram Halevy, 2018. "Piercing the 'Payoff Function' Veil: Tracing Beliefs and Motives," Working Papers tecipa-619, University of Toronto, Department of Economics.
    10. Sorensen, Morten, 2007. "Learning by Investing: Evidence from Venture Capital," SIFR Research Report Series 53, Institute for Financial Research.
    11. Giovanna Devetag & Sibilla Guida & Luca Polonio, 2016. "An eye-tracking study of feature-based choice in one-shot games," Experimental Economics, Springer;Economic Science Association, vol. 19(1), pages 177-201, March.
    12. Hu, Yingyao & Shum, Matthew, 2012. "Nonparametric identification of dynamic models with unobserved state variables," Journal of Econometrics, Elsevier, vol. 171(1), pages 32-44.
    13. Carrillo, Juan & Camerer, Colin & Brocas, Isabelle & Wang, Stephanie W., 2009. "Measuring attention and strategic behavior in games with private information," CEPR Discussion Papers 7529, C.E.P.R. Discussion Papers.
    14. Taisuke Imai & Min Jeong Kang & Colin F. Camerer, 2019. "When the eyes say buy: visual fixations during hypothetical consumer choice improve prediction of actual purchases," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 5(1), pages 112-122, August.
    15. Élise PAYZAN LE NESTOUR, 2010. "Bayesian Learning in UnstableSettings: Experimental Evidence Based on the Bandit Problem," Swiss Finance Institute Research Paper Series 10-28, Swiss Finance Institute.
    16. Georganas, Sotiris & Healy, Paul J. & Weber, Roberto A., 2015. "On the persistence of strategic sophistication," Journal of Economic Theory, Elsevier, vol. 159(PA), pages 369-400.
    17. Matthew P. Taylor, 2017. "Information Acquisition Under Risky Conditions Across Real And Hypothetical Settings," Economic Inquiry, Western Economic Association International, vol. 55(1), pages 352-367, January.
    18. Nagel, Rosemarie & Bühren, Christoph & Frank, Björn, 2017. "Inspired and inspiring: Hervé Moulin and the discovery of the beauty contest game," Mathematical Social Sciences, Elsevier, vol. 90(C), pages 191-207.
    19. Polonio, Luca & Coricelli, Giorgio, 2019. "Testing the level of consistency between choices and beliefs in games using eye-tracking," Games and Economic Behavior, Elsevier, vol. 113(C), pages 566-586.
    20. Marcoul, Philippe & Weninger, Quinn, 2008. "Search and active learning with correlated information: Empirical evidence from mid-Atlantic clam fishermen," Journal of Economic Dynamics and Control, Elsevier, vol. 32(6), pages 1921-1948, June.

    More about this item

    JEL classification:

    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:jhu:papers:560. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Humphrey Muturi (email available below). General contact details of provider: https://edirc.repec.org/data/dejhuus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.