IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2206.09235.html
   My bibliography  Save this paper

Risk Filtering and Risk-Averse Control of Markovian Systems Subject to Model Uncertainty

Author

Listed:
  • Tomasz R. Bielecki
  • Igor Cialenco
  • Andrzej Ruszczy'nski

Abstract

We consider a Markov decision process subject to model uncertainty in a Bayesian framework, where we assume that the state process is observed but its law is unknown to the observer. In addition, while the state process and the controls are observed at time $t$, the actual cost that may depend on the unknown parameter is not known at time $t$. The controller optimizes total cost by using a family of special risk measures, that we call risk filters and that are appropriately defined to take into account the model uncertainty of the controlled system. These key features lead to non-standard and non-trivial risk-averse control problems, for which we derive the Bellman principle of optimality. We illustrate the general theory on two practical examples: optimal investment and clinical trials.

Suggested Citation

  • Tomasz R. Bielecki & Igor Cialenco & Andrzej Ruszczy'nski, 2022. "Risk Filtering and Risk-Averse Control of Markovian Systems Subject to Model Uncertainty," Papers 2206.09235, arXiv.org.
    • Handle: RePEc:arx:papers:2206.09235
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2206.09235
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Nicole Bäuerle & Ulrich Rieder, 2014. "More Risk-Sensitive Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 39(1), pages 105-120, February.
    2. Bäuerle, Nicole & Rieder, Ulrich, 2017. "Zero-sum risk-sensitive stochastic games," Stochastic Processes and their Applications, Elsevier, vol. 127(2), pages 622-642.
    3. Mark H A Davis & Sébastien Lleo, 2014. "Risk-Sensitive Investment Management," World Scientific Books, World Scientific Publishing Co. Pte. Ltd., number 9026.
    4. Jingnan Fan & Andrzej Ruszczyński, 2018. "Risk measurement and risk-averse control of partially observable discrete-time Markov systems," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 88(2), pages 161-184, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marlon Moresco & M'elina Mailhot & Silvana M. Pesenti, 2023. "Uncertainty Propagation and Dynamic Robust Risk Measures," Papers 2308.12856, arXiv.org, revised Feb 2024.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Julio Saucedo-Zul & Rolando Cavazos-Cadena & Hugo Cruz-Suárez, 2020. "A Discounted Approach in Communicating Average Markov Decision Chains Under Risk-Aversion," Journal of Optimization Theory and Applications, Springer, vol. 187(2), pages 585-606, November.
    2. Ghosh, Mrinal K. & Golui, Subrata & Pal, Chandan & Pradhan, Somnath, 2023. "Discrete-time zero-sum games for Markov chains with risk-sensitive average cost criterion," Stochastic Processes and their Applications, Elsevier, vol. 158(C), pages 40-74.
    3. Qingda Wei & Xian Chen, 2023. "Continuous-Time Markov Decision Processes Under the Risk-Sensitive First Passage Discounted Cost Criterion," Journal of Optimization Theory and Applications, Springer, vol. 197(1), pages 309-333, April.
    4. O. L. V. Costa & F. Dufour, 2021. "Integro-differential optimality equations for the risk-sensitive control of piecewise deterministic Markov processes," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 93(2), pages 327-357, April.
    5. Bhabak, Arnab & Saha, Subhamay, 2022. "Risk-sensitive semi-Markov decision problems with discounted cost and general utilities," Statistics & Probability Letters, Elsevier, vol. 184(C).
    6. Dietmar Leisen & Eckhard Platen, 2017. "Investing for the Long Run," Papers 1705.03929, arXiv.org.
    7. Bäuerle, Nicole & Rieder, Ulrich, 2017. "Zero-sum risk-sensitive stochastic games," Stochastic Processes and their Applications, Elsevier, vol. 127(2), pages 622-642.
    8. Arvidsson, Björn & Johansson, Jonas & Guldåker, Nicklas, 2021. "Critical infrastructure, geographical information science and risk governance: A systematic cross-field review," Reliability Engineering and System Safety, Elsevier, vol. 213(C).
    9. Chen, Fang & Guo, Xianping, 2023. "Two-person zero-sum risk-sensitive stochastic games with incomplete reward information on one side," Stochastic Processes and their Applications, Elsevier, vol. 165(C), pages 218-245.
    10. Constantin Waubert de Puiseau & Richard Meyes & Tobias Meisen, 2022. "On reliability of reinforcement learning based production scheduling systems: a comparative survey," Journal of Intelligent Manufacturing, Springer, vol. 33(4), pages 911-927, April.
    11. Jan Obłój & Thaleia Zariphopoulou, 2021. "In memoriam: Mark H. A. Davis and his contributions to mathematical finance," Mathematical Finance, Wiley Blackwell, vol. 31(4), pages 1099-1110, October.
    12. Hubert Asienkiewicz & Łukasz Balbus, 2019. "Existence of Nash equilibria in stochastic games of resource extraction with risk-sensitive players," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 27(3), pages 502-518, October.
    13. Carlos Camilo-Garay & Rolando Cavazos-Cadena & Hugo Cruz-Suárez, 2022. "Contractive Approximations in Risk-Sensitive Average Semi-Markov Decision Chains on a Finite State Space," Journal of Optimization Theory and Applications, Springer, vol. 192(1), pages 271-291, January.
    14. Bäuerle, Nicole & Jaśkiewicz, Anna, 2017. "Optimal dividend payout model with risk sensitive preferences," Insurance: Mathematics and Economics, Elsevier, vol. 73(C), pages 82-93.
    15. Qingda Wei & Xian Chen, 2021. "Nonzero-sum Risk-Sensitive Average Stochastic Games: The Case of Unbounded Costs," Dynamic Games and Applications, Springer, vol. 11(4), pages 835-862, December.
    16. Lijun Bo & Huafu Liao & Xiang Yu, 2017. "Risk Sensitive Portfolio Optimization with Default Contagion and Regime-Switching," Papers 1712.05676, arXiv.org, revised Oct 2018.
    17. Adam Jonsson, 2023. "An axiomatic approach to Markov decision processes," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 97(1), pages 117-133, February.
    18. Mark H.A. Davis & Sébastien Lleo, 2021. "Risk‐sensitive benchmarked asset management with expert forecasts," Mathematical Finance, Wiley Blackwell, vol. 31(4), pages 1162-1189, October.
    19. E. Boguslavskaya & M. Boguslavsky & D. Muravey, 2020. "Trading multiple mean reversion," Papers 2009.09816, arXiv.org.
    20. Naci Saldi & Tamer Bas¸ ar & Maxim Raginsky, 2020. "Approximate Markov-Nash Equilibria for Discrete-Time Risk-Sensitive Mean-Field Games," Mathematics of Operations Research, INFORMS, vol. 45(4), pages 1596-1620, November.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2206.09235. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.