IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2104.00102.html
   My bibliography  Save this paper

Robust Experimentation in the Continuous Time Bandit Problem

Author

Listed:
  • Farzad Pourbabaee

Abstract

We study the experimentation dynamics of a decision maker (DM) in a two-armed bandit setup (Bolton and Harris (1999)), where the agent holds ambiguous beliefs regarding the distribution of the return process of one arm and is certain about the other one. The DM entertains Multiplier preferences a la Hansen and Sargent (2001), thus we frame the decision making environment as a two-player differential game against nature in continuous time. We characterize the DM value function and her optimal experimentation strategy that turns out to follow a cut-off rule with respect to her belief process. The belief threshold for exploring the ambiguous arm is found in closed form and is shown to be increasing with respect to the ambiguity aversion index. We then study the effect of provision of an unambiguous information source about the ambiguous arm. Interestingly, we show that the exploration threshold rises unambiguously as a result of this new information source, thereby leading to more conservatism. This analysis also sheds light on the efficient time to reach for an expert opinion.

Suggested Citation

  • Farzad Pourbabaee, 2021. "Robust Experimentation in the Continuous Time Bandit Problem," Papers 2104.00102, arXiv.org.
    • Handle: RePEc:arx:papers:2104.00102
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2104.00102
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Frank Riedel, 2009. "Optimal Stopping With Multiple Priors," Econometrica, Econometric Society, vol. 77(3), pages 857-908, May.
    2. Gustavo Manso, 2011. "Motivating Innovation," Journal of Finance, American Finance Association, vol. 66(5), pages 1823-1860, October.
    3. Bonatti, Alessandro & Hörner, Johannes, 2017. "Learning to disagree in a game of experimentation," Journal of Economic Theory, Elsevier, vol. 169(C), pages 234-269.
    4. Godfrey Keller & Sven Rady, 1999. "Optimal Experimentation in a Changing Environment," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 66(3), pages 475-507.
    5. Weitzman, Martin L, 1979. "Optimal Search for the Best Alternative," Econometrica, Econometric Society, vol. 47(3), pages 641-654, May.
    6. Fabio Maccheroni & Massimo Marinacci & Aldo Rustichini, 2006. "Ambiguity Aversion, Robustness, and the Variational Representation of Preferences," Econometrica, Econometric Society, vol. 74(6), pages 1447-1498, November.
    7. Lars Peter Hansen & Thomas J Sargent, 2014. "Robust Control and Model Uncertainty," World Scientific Book Chapters, in: UNCERTAINTY WITHIN ECONOMIC MODELS, chapter 5, pages 145-154, World Scientific Publishing Co. Pte. Ltd..
    8. Godfrey Keller & Sven Rady & Martin Cripps, 2005. "Strategic Experimentation with Exponential Bandits," Econometrica, Econometric Society, vol. 73(1), pages 39-68, January.
    9. Yaoyao Wu & Jinqiang Yang & Zhentao Zou, 2018. "Ambiguity sharing and the lack of relative performance evaluation," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 66(1), pages 141-157, July.
    10. Jianjun Miao & Alejandro Rivera, 2016. "Robust Contracts in Continuous Time," Econometrica, Econometric Society, vol. 84, pages 1405-1440, July.
    11. Epstein, Larry G. & Schneider, Martin, 2003. "Recursive multiple-priors," Journal of Economic Theory, Elsevier, vol. 113(1), pages 1-31, November.
    12. Lars Peter Hansen & Thomas J Sargent, 2014. "Robust Control and Model Misspecification," World Scientific Book Chapters, in: UNCERTAINTY WITHIN ECONOMIC MODELS, chapter 6, pages 155-216, World Scientific Publishing Co. Pte. Ltd..
    13. Larry G. Epstein & Martin Schneider, 2007. "Learning Under Ambiguity," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 74(4), pages 1275-1303.
    14. Patrick Bolton & Christopher Harris, 1999. "Strategic Experimentation," Econometrica, Econometric Society, vol. 67(2), pages 349-374, March.
    15. Christopher Anderson, 2012. "Ambiguity aversion in multi-armed bandit problems," Theory and Decision, Springer, vol. 72(1), pages 15-33, January.
    16. Robert J. Meyer & Yong Shi, 1995. "Sequential Choice Under Ambiguity: Intuitive Solutions to the Armed-Bandit Problem," Management Science, INFORMS, vol. 41(5), pages 817-834, May.
    17. Hansen, Lars Peter & Sargent, Thomas J., 2011. "Robustness and ambiguity in continuous time," Journal of Economic Theory, Elsevier, vol. 146(3), pages 1195-1223, May.
    18. Heidhues, Paul & Rady, Sven & Strack, Philipp, 2015. "Strategic experimentation with private payoffs," Journal of Economic Theory, Elsevier, vol. 159(PA), pages 531-551.
    19. Maccheroni, Fabio & Marinacci, Massimo & Rustichini, Aldo, 2006. "Dynamic variational preferences," Journal of Economic Theory, Elsevier, vol. 128(1), pages 4-44, May.
    20. Larry G. Epstein & Shaolin Ji, 2022. "Optimal Learning Under Robustness and Time-Consistency," Operations Research, INFORMS, vol. 70(3), pages 1317-1329, May.
    21. Gilboa, Itzhak & Schmeidler, David, 1989. "Maxmin expected utility with non-unique prior," Journal of Mathematical Economics, Elsevier, vol. 18(2), pages 141-153, April.
    22. Massimo Marinacci, 2002. "Learning from ambiguous urns," Statistical Papers, Springer, vol. 43(1), pages 143-151, January.
    23. Li, Jian, 2019. "The K-armed bandit problem with multiple priors," Journal of Mathematical Economics, Elsevier, vol. 80(C), pages 22-38.
    24. Yulei Luo, 2017. "Robustly Strategic Consumption–Portfolio Rules with Informational Frictions," Management Science, INFORMS, vol. 63(12), pages 4158-4174, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Farzad Pourbabaee, 2022. "Robust experimentation in the continuous time bandit problem," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 73(1), pages 151-181, February.
    2. Li, Jian, 2019. "The K-armed bandit problem with multiple priors," Journal of Mathematical Economics, Elsevier, vol. 80(C), pages 22-38.
    3. Hansen, Peter G., 2022. "New formulations of ambiguous volatility with an application to optimal dynamic contracting," Journal of Economic Theory, Elsevier, vol. 199(C).
    4. Li, Jing, 2018. "Essays on model uncertainty in financial models," Other publications TiSEM 202cd910-7ef1-4db4-94ae-d, Tilburg University, School of Economics and Management.
    5. Hansen, Lars Peter & Sargent, Thomas J., 2022. "Structured ambiguity and model misspecification," Journal of Economic Theory, Elsevier, vol. 199(C).
    6. Hansen, Lars Peter & Szőke, Bálint & Han, Lloyd S. & Sargent, Thomas J., 2020. "Twisted probabilities, uncertainty, and prices," Journal of Econometrics, Elsevier, vol. 216(1), pages 151-174.
    7. Chambers, Robert G. & Melkonyan, Tigran, 2009. "Smoothing preference kinks with information," Mathematical Social Sciences, Elsevier, vol. 58(2), pages 173-189, September.
    8. Peter G. Hansen, 2021. "New Formulations of Ambiguous Volatility with an Application to Optimal Dynamic Contracting," Papers 2101.12306, arXiv.org.
    9. Swagata Bhattacharjee, 2019. "Dynamic Contracting for Innovation Under Ambiguity," Working Papers 1022, Ashoka University, Department of Economics, revised Aug 2019.
    10. Battigalli, P. & Francetich, A. & Lanzani, G. & Marinacci, M., 2019. "Learning and self-confirming long-run biases," Journal of Economic Theory, Elsevier, vol. 183(C), pages 740-785.
    11. Daniele Pennesi, 2013. "Asset Prices in an Ambiguous Economy," Carlo Alberto Notebooks 315, Collegio Carlo Alberto.
    12. Paul Viefers, 2012. "Should I Stay or Should I Go?: A Laboratory Analysis of Investment Opportunities under Ambiguity," Discussion Papers of DIW Berlin 1228, DIW Berlin, German Institute for Economic Research.
    13. Bhattacharjee, Swagata, 2022. "Dynamic contracting for innovation under ambiguity," Games and Economic Behavior, Elsevier, vol. 132(C), pages 534-552.
    14. Li, Jian & Zhou, Junjie, 2016. "Blackwell's informativeness ranking with uncertainty-averse preferences," Games and Economic Behavior, Elsevier, vol. 96(C), pages 18-29.
    15. Alexander Zimper, 2011. "Do Bayesians Learn Their Way Out of Ambiguity?," Decision Analysis, INFORMS, vol. 8(4), pages 269-285, December.
    16. Michael Barnett & Greg Buchak & Constantine Yannelis, 2023. "Epidemic responses under uncertainty," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 120(2), pages 2208111120-, January.
    17. Agarwal, Vikas & Arisoy, Y. Eser & Naik, Narayan Y., 2017. "Volatility of aggregate volatility and hedge fund returns," Journal of Financial Economics, Elsevier, vol. 125(3), pages 491-510.
    18. Massimo Guidolin & Francesca Rinaldi, 2013. "Ambiguity in asset pricing and portfolio choice: a review of the literature," Theory and Decision, Springer, vol. 74(2), pages 183-217, February.
    19. Nengjiu Ju & Jianjun Miao, 2012. "Ambiguity, Learning, and Asset Returns," Econometrica, Econometric Society, vol. 80(2), pages 559-591, March.
    20. Bommier, Antoine & Kochov, Asen & Le Grand, François, 2019. "Ambiguity and endogenous discounting," Journal of Mathematical Economics, Elsevier, vol. 83(C), pages 48-62.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2104.00102. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.