IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1907.05689.html
   My bibliography  Save this paper

Gittins' theorem under uncertainty

Author

Listed:
  • Samuel N. Cohen
  • Tanut Treetanthiploet

Abstract

We study dynamic allocation problems for discrete time multi-armed bandits under uncertainty, based on the the theory of nonlinear expectations. We show that, under strong independence of the bandits and with some relaxation in the definition of optimality, a Gittins allocation index gives optimal choices. This involves studying the interaction of our uncertainty with controls which determine the filtration. We also run a simple numerical example which illustrates the interaction between the willingness to explore and uncertainty aversion of the agent when making decisions.

Suggested Citation

  • Samuel N. Cohen & Tanut Treetanthiploet, 2019. "Gittins' theorem under uncertainty," Papers 1907.05689, arXiv.org, revised Jun 2021.
    • Handle: RePEc:arx:papers:1907.05689
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1907.05689
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Marco Frittelli & Giacomo Scandolo, 2006. "Risk Measures And Capital Requirements For Processes," Mathematical Finance, Wiley Blackwell, vol. 16(4), pages 589-612, October.
    2. Riedel, Frank, 2004. "Dynamic coherent risk measures," Stochastic Processes and their Applications, Elsevier, vol. 112(2), pages 185-200, August.
    3. Frank Riedel, 2009. "Optimal Stopping With Multiple Priors," Econometrica, Econometric Society, vol. 77(3), pages 857-908, May.
    4. Frittelli, Marco & Rosazza Gianin, Emanuela, 2002. "Putting order in risk measures," Journal of Banking & Finance, Elsevier, vol. 26(7), pages 1473-1486, July.
    5. Ying Hu & Hanqing Jin & Xun Yu Zhou, 2012. "Time-Inconsistent Stochastic Linear--Quadratic Control," Post-Print hal-00691816, HAL.
    6. Bank, Peter & Küchler, Christian, 2007. "On Gittins' index theorem in continuous time," Stochastic Processes and their Applications, Elsevier, vol. 117(9), pages 1357-1371, September.
    7. Jocelyne Bion-Nadal, 2008. "Dynamic risk measures: Time consistency and risk measures from BMO martingales," Finance and Stochastics, Springer, vol. 12(2), pages 219-244, April.
    8. R. H. Strotz, 1955. "Myopia and Inconsistency in Dynamic Utility Maximization," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 23(3), pages 165-180.
    9. Samuel N. Cohen, 2017. "Data and uncertainty in extreme risks - a nonlinear expectations approach," Papers 1705.08301, arXiv.org, revised Feb 2018.
    10. Daniel Kahneman & Amos Tversky, 2013. "Prospect Theory: An Analysis of Decision Under Risk," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 6, pages 99-127, World Scientific Publishing Co. Pte. Ltd..
    11. Tomas Björk & Agatha Murgoci, 2014. "A theory of Markovian time-inconsistent stochastic control in discrete time," Finance and Stochastics, Springer, vol. 18(3), pages 545-592, July.
    12. Bezalel Peleg & Menahem E. Yaari, 1973. "On the Existence of a Consistent Course of Action when Tastes are Changing," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 40(3), pages 391-401.
    13. Esther Frostig & Gideon Weiss, 2016. "Four proofs of Gittins’ multiarmed bandit theorem," Annals of Operations Research, Springer, vol. 241(1), pages 127-165, June.
    14. Brezzi, Monica & Lai, Tze Leung, 2002. "Optimal learning and experimentation in bandit problems," Journal of Economic Dynamics and Control, Elsevier, vol. 27(1), pages 87-108, November.
    15. N. El Karoui & S. Peng & M. C. Quenez, 1997. "Backward Stochastic Differential Equations in Finance," Mathematical Finance, Wiley Blackwell, vol. 7(1), pages 1-71, January.
    16. Philippe Artzner & Freddy Delbaen & Jean‐Marc Eber & David Heath, 1999. "Coherent Measures of Risk," Mathematical Finance, Wiley Blackwell, vol. 9(3), pages 203-228, July.
    17. Cohen, Samuel N. & Elliott, Robert J., 2010. "A general theory of finite state Backward Stochastic Difference Equations," Stochastic Processes and their Applications, Elsevier, vol. 120(4), pages 442-466, April.
    18. Kai Detlefsen & Giacomo Scandolo, 2005. "Conditional and dynamic convex risk measures," Finance and Stochastics, Springer, vol. 9(4), pages 539-561, October.
    19. Samuel N. Cohen, 2016. "Data-driven nonlinear expectations for statistical uncertainty in decisions," Papers 1609.06545, arXiv.org.
    20. Quiggin, John, 1982. "A theory of anticipated utility," Journal of Economic Behavior & Organization, Elsevier, vol. 3(4), pages 323-343, December.
    21. Garud N. Iyengar, 2005. "Robust Dynamic Programming," Mathematics of Operations Research, INFORMS, vol. 30(2), pages 257-280, May.
    22. Tomas Björk & Mariana Khapko & Agatha Murgoci, 2017. "On time-inconsistent stochastic control in continuous time," Finance and Stochastics, Springer, vol. 21(2), pages 331-360, April.
    23. Xiaoguang Huo & Feng Fu, 2017. "Risk-Aware Multi-Armed Bandit Problem with Application to Portfolio Selection," Papers 1709.04415, arXiv.org.
    24. Kai Detlefsen & Giacomo Scandolo, 2005. "Conditional and Dynamic Convex Risk Measures," SFB 649 Discussion Papers SFB649DP2005-006, Sonderforschungsbereich 649, Humboldt University, Berlin, Germany.
    25. Arnab Nilim & Laurent El Ghaoui, 2005. "Robust Control of Markov Decision Processes with Uncertain Transition Matrices," Operations Research, INFORMS, vol. 53(5), pages 780-798, October.
    26. Philippe Artzner & Freddy Delbaen & Jean-Marc Eber & David Heath & Hyejin Ku, 2007. "Coherent multiperiod risk adjusted values and Bellman’s principle," Annals of Operations Research, Springer, vol. 152(1), pages 5-22, July.
    27. Samuel N. Cohen, 2018. "Data and Uncertainty in Extreme Risks: A Nonlinear Expectations Approach," World Scientific Book Chapters, in: Kathrin Glau & Daniël Linders & Aleksey Min & Matthias Scherer & Lorenz Schneider & Rudi Zagst (ed.), Innovations in Insurance, Risk- and Asset Management, chapter 6, pages 135-162, World Scientific Publishing Co. Pte. Ltd..
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zachary Feinstein & Birgit Rudloff, 2018. "Scalar multivariate risk measures with a single eligible asset," Papers 1807.10694, arXiv.org, revised Feb 2021.
    2. Acciaio, Beatrice & Föllmer, Hans & Penner, Irina, 2012. "Risk assessment for uncertain cash flows: model ambiguity, discounting ambiguity, and the role of bubbles," LSE Research Online Documents on Economics 50118, London School of Economics and Political Science, LSE Library.
    3. Zachary Feinstein & Birgit Rudloff, 2018. "Time consistency for scalar multivariate risk measures," Papers 1810.04978, arXiv.org, revised Nov 2021.
    4. Ji, Ronglin & Shi, Xuejun & Wang, Shijie & Zhou, Jinming, 2019. "Dynamic risk measures for processes via backward stochastic differential equations," Insurance: Mathematics and Economics, Elsevier, vol. 86(C), pages 43-50.
    5. Dan A. Iancu & Marek Petrik & Dharmashankar Subramanian, 2015. "Tight Approximations of Dynamic Risk Measures," Mathematics of Operations Research, INFORMS, vol. 40(3), pages 655-682, March.
    6. Zachary Feinstein & Birgit Rudloff, 2012. "Multiportfolio time consistency for set-valued convex and coherent risk measures," Papers 1212.5563, arXiv.org, revised Oct 2014.
    7. Wayne King Ming Chan, 2015. "RAROC-Based Contingent Claim Valuation," PhD Thesis, Finance Discipline Group, UTS Business School, University of Technology, Sydney, number 3-2015.
    8. Nicole EL KAROUI & Claudia RAVANELLI, 2008. "Cash Sub-additive Risk Measures and Interest Rate Ambiguity," Swiss Finance Institute Research Paper Series 08-09, Swiss Finance Institute.
    9. Xue Dong He & Xun Yu Zhou, 2021. "Who Are I: Time Inconsistency and Intrapersonal Conflict and Reconciliation," Papers 2105.01829, arXiv.org.
    10. Wing Fung Chong & Ying Hu & Gechun Liang & Thaleia Zariphopoulou, 2019. "An ergodic BSDE approach to forward entropic risk measures: representation and large-maturity behavior," Finance and Stochastics, Springer, vol. 23(1), pages 239-273, January.
    11. Henri Gérard & Michel Lara & Jean-Philippe Chancelier, 2020. "Equivalence between time consistency and nested formula," Annals of Operations Research, Springer, vol. 292(2), pages 627-647, September.
    12. Daniel Bartl, 2016. "Conditional nonlinear expectations," Papers 1612.09103, arXiv.org, revised Mar 2019.
    13. Dejian Tian & Xunlian Wang, 2023. "Dynamic star-shaped risk measures and $g$-expectations," Papers 2305.02481, arXiv.org.
    14. Roorda Berend & Schumacher Hans, 2013. "Membership conditions for consistent families of monetary valuations," Statistics & Risk Modeling, De Gruyter, vol. 30(3), pages 255-280, August.
    15. Beatrice Acciaio & Hans Föllmer & Irina Penner, 2012. "Risk assessment for uncertain cash flows: model ambiguity, discounting ambiguity, and the role of bubbles," Finance and Stochastics, Springer, vol. 16(4), pages 669-709, October.
    16. Beatrice Acciaio & Irina Penner, 2010. "Dynamic risk measures," Papers 1002.3794, arXiv.org.
    17. Beatrice Acciaio & Hans Foellmer & Irina Penner, 2010. "Risk assessment for uncertain cash flows: Model ambiguity, discounting ambiguity, and the role of bubbles," Papers 1002.3627, arXiv.org.
    18. Yanhong Chen & Zachary Feinstein, 2022. "Set-valued dynamic risk measures for processes and for vectors," Finance and Stochastics, Springer, vol. 26(3), pages 505-533, July.
    19. Freddy Delbaen & Shige Peng & Emanuela Rosazza Gianin, 2010. "Representation of the penalty term of dynamic concave utilities," Finance and Stochastics, Springer, vol. 14(3), pages 449-472, September.
    20. Elisa Mastrogiacomo & Emanuela Rosazza Gianin, 2019. "Time-consistency of risk measures: how strong is such a property?," Decisions in Economics and Finance, Springer;Associazione per la Matematica, vol. 42(1), pages 287-317, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1907.05689. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.