IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2112.10993.html
   My bibliography  Save this paper

Learning in Random Utility Models Via Online Decision Problems

Author

Listed:
  • Emerson Melo

Abstract

This paper studies the Random Utility Model (RUM) in a repeated stochastic choice situation, in which the decision maker is imperfectly informed about the payoffs of each available alternative. We develop a gradient-based learning algorithm by embedding the RUM into an online decision problem. We show that a large class of RUMs are Hannan consistent (\citet{Hahn1957}); that is, the average difference between the expected payoffs generated by a RUM and that of the best-fixed policy in hindsight goes to zero as the number of periods increase. In addition, we show that our gradient-based algorithm is equivalent to the Follow the Regularized Leader (FTRL) algorithm, which is widely used in the machine learning literature to model learning in repeated stochastic choice problems. Thus, we provide an economically grounded optimization framework to the FTRL algorithm. Finally, we apply our framework to study recency bias, no-regret learning in normal form games, and prediction markets.

Suggested Citation

  • Emerson Melo, 2021. "Learning in Random Utility Models Via Online Decision Problems," Papers 2112.10993, arXiv.org, revised Aug 2022.
  • Handle: RePEc:arx:papers:2112.10993
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2112.10993
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Andrew Caplin & Daniel Martin, 2015. "A Testable Theory of Imperfect Perception," Economic Journal, Royal Economic Society, vol. 125(582), pages 184-202, February.
    2. Manski, Charles F., 2006. "Interpreting the predictions of prediction markets," Economics Letters, Elsevier, vol. 91(3), pages 425-429, June.
    3. Bergemann, Dirk & Morris, Stephen, 2016. "Bayes correlated equilibrium and the comparison of information structures in games," Theoretical Economics, Econometric Society, vol. 11(2), May.
    4. Mogens Fosgerau & Emerson Melo & André de Palma & Matthew Shum, 2020. "Discrete Choice And Rational Inattention: A General Equivalence Result," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 61(4), pages 1569-1589, November.
    5. David E. Bell, 1982. "Regret in Decision Making under Uncertainty," Operations Research, INFORMS, vol. 30(5), pages 961-981, October.
    6. Fudenberg, Drew & Levine, David, 1998. "Learning in games," European Economic Review, Elsevier, vol. 42(3-5), pages 631-639, May.
    7. Foster, Dean P. & Vohra, Rakesh V., 1997. "Calibrated Learning and Correlated Equilibrium," Games and Economic Behavior, Elsevier, vol. 21(1-2), pages 40-55, October.
    8. Small, Kenneth A, 1987. "A Discrete Choice Model for Ordered Alternatives," Econometrica, Econometric Society, vol. 55(2), pages 409-424, March.
    9. Train,Kenneth E., 2009. "Discrete Choice Methods with Simulation," Cambridge Books, Cambridge University Press, number 9780521766555, November.
    10. Filip Matêjka & Alisdair McKay, 2015. "Rational Inattention to Discrete Choices: A New Foundation for the Multinomial Logit Model," American Economic Review, American Economic Association, vol. 105(1), pages 272-298, January.
    11. S. Cerreia-Vioglio & F. Maccheroni & M. Marinacci & A. Rustichini, 2017. "Multinomial logit processes and preference discovery: inside and outside the black box," Working Papers 615, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    12. Gualdani, Cristina & Sinha, Shruti, 2019. "Identification and inference in discrete choice models with imperfect information," TSE Working Papers 19-1049, Toulouse School of Economics (TSE), revised Jun 2020.
    13. Foster, Dean P. & Vohra, Rakesh, 1999. "Regret in the On-Line Decision Problem," Games and Economic Behavior, Elsevier, vol. 29(1-2), pages 7-35, October.
    14. Drew Fudenberg & Ryota Iijima & Tomasz Strzalecki, 2015. "Stochastic Choice and Revealed Perturbed Utility," Econometrica, Econometric Society, vol. 83, pages 2371-2409, November.
    15. Sergiu Hart & Andreu Mas-Colell, 2013. "A Simple Adaptive Procedure Leading To Correlated Equilibrium," World Scientific Book Chapters, in: Simple Adaptive Strategies From Regret-Matching to Uncoupled Dynamics, chapter 2, pages 17-46, World Scientific Publishing Co. Pte. Ltd..
    16. Fudenberg, Drew & Levine, David K., 1995. "Consistency and cautious fictitious play," Journal of Economic Dynamics and Control, Elsevier, vol. 19(5-7), pages 1065-1089.
    17. Paulo Natenzon, 2019. "Random Choice and Learning," Journal of Political Economy, University of Chicago Press, vol. 127(1), pages 419-457.
    18. Loomes, Graham & Sugden, Robert, 1982. "Regret Theory: An Alternative Theory of Rational Choice under Uncertainty," Economic Journal, Royal Economic Society, vol. 92(368), pages 805-824, December.
    19. McKelvey Richard D. & Palfrey Thomas R., 1995. "Quantal Response Equilibria for Normal Form Games," Games and Economic Behavior, Elsevier, vol. 10(1), pages 6-38, July.
    20. Josef Hofbauer & William H. Sandholm, 2002. "On the Global Convergence of Stochastic Fictitious Play," Econometrica, Econometric Society, vol. 70(6), pages 2265-2294, November.
    21. Todd Sarver, 2008. "Anticipating Regret: Why Fewer Options May Be Better," Econometrica, Econometric Society, vol. 76(2), pages 263-305, March.
    22. David Muller & Yurii Nesterov & Vladimir Shikhman, 2019. "Discrete choice prox-functions on the simplex," Papers 1909.05591, arXiv.org.
    23. H.D. Block & Jacob Marschak, 1959. "Random Orderings and Stochastic Theories of Response," Cowles Foundation Discussion Papers 66, Cowles Foundation for Research in Economics, Yale University.
    24. Alfred Galichon & Bernard Salani'e, 2021. "Cupid's Invisible Hand: Social Surplus and Identification in Matching Models," Papers 2106.02371, arXiv.org, revised Jan 2023.
    25. Marina Agranov & Pietro Ortoleva, 2017. "Stochastic Choice and Preferences for Randomization," Journal of Political Economy, University of Chicago Press, vol. 125(1), pages 40-68.
    26. Drew Fudenberg & Peysakhovich, A, 2014. "Recency, Records and Recaps: Learning and Non-Equilibrium Behavior in a Simple Decision Problem," Working Paper 167691, Harvard University OpenScholar.
    27. Wen, Chieh-Hua & Koppelman, Frank S., 2001. "The generalized nested logit model," Transportation Research Part B: Methodological, Elsevier, vol. 35(7), pages 627-641, August.
    28. Guiyun Feng & Xiaobo Li & Zizhuo Wang, 2017. "Technical Note—On the Relation Between Several Discrete Choice Models," Operations Research, INFORMS, vol. 65(6), pages 1516-1525, December.
    29. Sørensen, Jesper R.-V. & Fosgerau, Mogens, 2022. "How McFadden met Rockafellar and learned to do more with less," Journal of Mathematical Economics, Elsevier, vol. 100(C).
    30. Daniel McFadden, 2001. "Economic Choices," American Economic Review, American Economic Association, vol. 91(3), pages 351-378, June.
    31. Jacob Marschak, 1959. "Binary Choice Constraints on Random Utility Indicators," Cowles Foundation Discussion Papers 74, Cowles Foundation for Research in Economics, Yale University.
    32. repec:cup:cbooks:9781316779309 is not listed on IDEAS
    33. Cristina Gualdani & Shruti Sinha, 2019. "Identification in discrete choice models with imperfect information," Papers 1911.04529, arXiv.org, revised Dec 2023.
    34. Han Bleichrodt & Peter P. Wakker, 2015. "Regret Theory: A Bold Alternative to the Alternatives," Economic Journal, Royal Economic Society, vol. 0(583), pages 493-532, March.
    35. Roughgarden,Tim, 2016. "Twenty Lectures on Algorithmic Game Theory," Cambridge Books, Cambridge University Press, number 9781316624791, November.
    36. Drew Fudenberg & David K. Levine, 1998. "The Theory of Learning in Games," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262061945, December.
    37. Roughgarden,Tim, 2016. "Twenty Lectures on Algorithmic Game Theory," Cambridge Books, Cambridge University Press, number 9781107172661, November.
    38. repec:hal:pseose:halshs-01155313 is not listed on IDEAS
    39. Freund, Yoav & Schapire, Robert E., 1999. "Adaptive Game Playing Using Multiplicative Weights," Games and Economic Behavior, Elsevier, vol. 29(1-2), pages 79-103, October.
    40. Loomes, Graham & Sugden, Robert, 1987. "Some implications of a more general form of regret theory," Journal of Economic Theory, Elsevier, vol. 41(2), pages 270-287, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Emerson Melo, 2021. "Learning In Random Utility Models Via Online Decision Problems," CAEPR Working Papers 2022-003 Classification-D, Center for Applied Economics and Policy Research, Department of Economics, Indiana University Bloomington.
    2. Emerson Melo, 2022. "On the uniqueness of quantal response equilibria and its application to network games," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 74(3), pages 681-725, October.
    3. Michel Benaïm & Josef Hofbauer & Sylvain Sorin, 2006. "Stochastic Approximations and Differential Inclusions, Part II: Applications," Mathematics of Operations Research, INFORMS, vol. 31(4), pages 673-695, November.
    4. Sergiu Hart & Andreu Mas-Colell, 2013. "A General Class Of Adaptive Strategies," World Scientific Book Chapters, in: Simple Adaptive Strategies From Regret-Matching to Uncoupled Dynamics, chapter 3, pages 47-76, World Scientific Publishing Co. Pte. Ltd..
    5. Tim Roughgarden, 2018. "Complexity Theory, Game Theory, and Economics: The Barbados Lectures," Papers 1801.00734, arXiv.org, revised Feb 2020.
    6. Mogens Fosgerau & Julien Monardo & André de Palma, 2019. "The Inverse Product Differentiation Logit Model," Working Papers hal-02183411, HAL.
    7. Duffy, Sean & Gussman, Steven & Smith, John, 2021. "Visual judgments of length in the economics laboratory: Are there brains in stochastic choice?," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 93(C).
    8. Eric Friedman & Scott Shenker & Amy Greenwald, 1998. "Learning in Networks Contexts: Experimental Results from Simulations," Departmental Working Papers 199825, Rutgers University, Department of Economics.
    9. Mogens Fosgerau & John Rehbeck, 2023. "Nontransitive Preferences and Stochastic Rationalizability: A Behavioral Equivalence," Papers 2304.14631, arXiv.org.
    10. Germano, Fabrizio & Lugosi, Gabor, 2007. "Global Nash convergence of Foster and Young's regret testing," Games and Economic Behavior, Elsevier, vol. 60(1), pages 135-154, July.
    11. Du, Ye & Lehrer, Ehud, 2020. "Constrained no-regret learning," Journal of Mathematical Economics, Elsevier, vol. 88(C), pages 16-24.
    12. Nicolò Cesa-Bianchi & Gábor Lugosi & Gilles Stoltz, 2006. "Regret Minimization Under Partial Monitoring," Mathematics of Operations Research, INFORMS, vol. 31(3), pages 562-580, August.
    13. Block, Juan I. & Fudenberg, Drew & Levine, David K., 2019. "Learning dynamics with social comparisons and limited memory," Theoretical Economics, Econometric Society, vol. 14(1), January.
    14. Xiaotie Deng & Xinyan Hu & Tao Lin & Weiqiang Zheng, 2021. "Nash Convergence of Mean-Based Learning Algorithms in First Price Auctions," Papers 2110.03906, arXiv.org, revised Feb 2023.
    15. Duffy, Sean & Smith, John, 2020. "An economist and a psychologist form a line: What can imperfect perception of length tell us about stochastic choice?," MPRA Paper 99417, University Library of Munich, Germany.
    16. Cominetti, Roberto & Melo, Emerson & Sorin, Sylvain, 2010. "A payoff-based learning procedure and its application to traffic games," Games and Economic Behavior, Elsevier, vol. 70(1), pages 71-83, September.
    17. Panayotis Mertikopoulos & William H. Sandholm, 2016. "Learning in Games via Reinforcement and Regularization," Mathematics of Operations Research, INFORMS, vol. 41(4), pages 1297-1324, November.
    18. Fudenberg, Drew & Levine, David K., 1999. "Conditional Universal Consistency," Games and Economic Behavior, Elsevier, vol. 29(1-2), pages 104-130, October.
    19. Tom Johnston & Michael Savery & Alex Scott & Bassel Tarbush, 2023. "Game Connectivity and Adaptive Dynamics," Papers 2309.10609, arXiv.org, revised Nov 2023.
    20. Benaïm, Michel & Hofbauer, Josef & Hopkins, Ed, 2009. "Learning in games with unstable equilibria," Journal of Economic Theory, Elsevier, vol. 144(4), pages 1694-1709, July.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2112.10993. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.