IDEAS home Printed from https://ideas.repec.org/a/spr/mathme/v94y2021i1d10.1007_s00186-021-00746-w.html
   My bibliography  Save this article

Minimizing spectral risk measures applied to Markov decision processes

Author

Listed:
  • Nicole Bäuerle

    (Karlsruhe Institute of Technology (KIT))

  • Alexander Glauner

    (Karlsruhe Institute of Technology (KIT))

Abstract

We study the minimization of a spectral risk measure of the total discounted cost generated by a Markov Decision Process (MDP) over a finite or infinite planning horizon. The MDP is assumed to have Borel state and action spaces and the cost function may be unbounded above. The optimization problem is split into two minimization problems using an infimum representation for spectral risk measures. We show that the inner minimization problem can be solved as an ordinary MDP on an extended state space and give sufficient conditions under which an optimal policy exists. Regarding the infinite dimensional outer minimization problem, we prove the existence of a solution and derive an algorithm for its numerical approximation. Our results include the findings in Bäuerle and Ott (Math Methods Oper Res 74(3):361–379, 2011) in the special case that the risk measure is Expected Shortfall. As an application, we present a dynamic extension of the classical static optimal reinsurance problem, where an insurance company minimizes its cost of capital.

Suggested Citation

  • Nicole Bäuerle & Alexander Glauner, 2021. "Minimizing spectral risk measures applied to Markov decision processes," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 94(1), pages 35-69, August.
    • Handle: RePEc:spr:mathme:v:94:y:2021:i:1:d:10.1007_s00186-021-00746-w
    DOI: 10.1007/s00186-021-00746-w
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00186-021-00746-w
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00186-021-00746-w?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Acerbi, Carlo, 2002. "Spectral measures of risk: A coherent representation of subjective risk aversion," Journal of Banking & Finance, Elsevier, vol. 26(7), pages 1505-1518, July.
    2. Alexander J. McNeil & Rüdiger Frey & Paul Embrechts, 2015. "Quantitative Risk Management: Concepts, Techniques and Tools Revised edition," Economics Books, Princeton University Press, edition 2, number 10496.
    3. Tsanakas, A. & Desli, E., 2003. "Risk Measures and Theories of Choice," British Actuarial Journal, Cambridge University Press, vol. 9(4), pages 959-991, October.
    4. Georg Ch. Pflug & Alois Pichler, 2016. "Time-Consistent Decisions and Temporal Decomposition of Coherent Risk Functionals," Mathematics of Operations Research, INFORMS, vol. 41(2), pages 682-699, May.
    5. Alexander Shapiro, 2013. "On Kusuoka Representation of Law Invariant Risk Measures," Mathematics of Operations Research, INFORMS, vol. 38(1), pages 142-152, February.
    6. Nicole Bäuerle & Jonathan Ott, 2011. "Markov Decision Processes with Average-Value-at-Risk criteria," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 74(3), pages 361-379, December.
    7. Cui, Wei & Yang, Jingping & Wu, Lan, 2013. "Optimal reinsurance minimizing the distortion risk measure under general reinsurance premium principles," Insurance: Mathematics and Economics, Elsevier, vol. 53(1), pages 74-85.
    8. Nicole Bäuerle & Ulrich Rieder, 2014. "More Risk-Sensitive Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 39(1), pages 105-120, February.
    9. Ronald A. Howard & James E. Matheson, 1972. "Risk-Sensitive Markov Decision Processes," Management Science, INFORMS, vol. 18(7), pages 356-369, March.
    10. Lo, Ambrose, 2017. "A Neyman-Pearson Perspective On Optimal Reinsurance With Constraints," ASTIN Bulletin, Cambridge University Press, vol. 47(2), pages 467-499, May.
    11. Bäuerle, Nicole & Glauner, Alexander, 2018. "Optimal risk allocation in reinsurance networks," Insurance: Mathematics and Economics, Elsevier, vol. 82(C), pages 37-47.
    12. Chi, Yichun & Tan, Ken Seng, 2013. "Optimal reinsurance with general premium principles," Insurance: Mathematics and Economics, Elsevier, vol. 52(2), pages 180-189.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ziteng Cheng & Sebastian Jaimungal, 2022. "Risk-Averse Markov Decision Processes through a Distributional Lens," Papers 2203.09612, arXiv.org, revised Apr 2024.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nicole Bauerle & Alexander Glauner, 2020. "Minimizing Spectral Risk Measures Applied to Markov Decision Processes," Papers 2012.04521, arXiv.org.
    2. Alexander Glauner, 2020. "Dynamic Reinsurance in Discrete Time Minimizing the Insurer's Cost of Capital," Papers 2012.09648, arXiv.org.
    3. Yichun Chi & Zuo Quan Xu & Sheng Chao Zhuang, 2022. "Distributionally Robust Goal-Reaching Optimization in the Presence of Background Risk," North American Actuarial Journal, Taylor & Francis Journals, vol. 26(3), pages 351-382, August.
    4. Liu, Fangda & Cai, Jun & Lemieux, Christiane & Wang, Ruodu, 2020. "Convex risk functionals: Representation and applications," Insurance: Mathematics and Economics, Elsevier, vol. 90(C), pages 66-79.
    5. Zhuang, Sheng Chao & Weng, Chengguo & Tan, Ken Seng & Assa, Hirbod, 2016. "Marginal Indemnification Function formulation for optimal reinsurance," Insurance: Mathematics and Economics, Elsevier, vol. 67(C), pages 65-76.
    6. Alois Pichler, 2017. "A quantitative comparison of risk measures," Annals of Operations Research, Springer, vol. 254(1), pages 251-275, July.
    7. Bäuerle, Nicole & Glauner, Alexander, 2022. "Markov decision processes with recursive risk measures," European Journal of Operational Research, Elsevier, vol. 296(3), pages 953-966.
    8. Birghila, Corina & Pflug, Georg Ch., 2019. "Optimal XL-insurance under Wasserstein-type ambiguity," Insurance: Mathematics and Economics, Elsevier, vol. 88(C), pages 30-43.
    9. Tomasz R. Bielecki & Igor Cialenco & Marcin Pitera & Thorsten Schmidt, 2019. "Fair Estimation of Capital Risk Allocation," Papers 1902.10044, arXiv.org, revised Nov 2019.
    10. Tsanakas, Andreas, 2009. "To split or not to split: Capital allocation with convex risk measures," Insurance: Mathematics and Economics, Elsevier, vol. 44(2), pages 268-277, April.
    11. Wei Wang & Huifu Xu, 2023. "Preference robust distortion risk measure and its application," Mathematical Finance, Wiley Blackwell, vol. 33(2), pages 389-434, April.
    12. Ruodu Wang & Yunran Wei, 2020. "Risk functionals with convex level sets," Mathematical Finance, Wiley Blackwell, vol. 30(4), pages 1337-1367, October.
    13. Tim J. Boonen, 2016. "Optimal Reinsurance with Heterogeneous Reference Probabilities," Risks, MDPI, vol. 4(3), pages 1-11, July.
    14. Yi Shen & Zachary Van Oosten & Ruodu Wang, 2024. "Partially Law-Invariant Risk Measures," Papers 2401.17265, arXiv.org.
    15. Wächter, Hans Peter & Mazzoni, Thomas, 2013. "Consistent modeling of risk averse behavior with spectral risk measures," European Journal of Operational Research, Elsevier, vol. 229(2), pages 487-495.
    16. Bäuerle, Nicole & Rieder, Ulrich, 2017. "Zero-sum risk-sensitive stochastic games," Stochastic Processes and their Applications, Elsevier, vol. 127(2), pages 622-642.
    17. James Ming Chen, 2018. "On Exactitude in Financial Regulation: Value-at-Risk, Expected Shortfall, and Expectiles," Risks, MDPI, vol. 6(2), pages 1-28, June.
    18. Mi Chen & Wenyuan Wang & Ruixing Ming, 2016. "Optimal Reinsurance Under General Law-Invariant Convex Risk Measure and TVaR Premium Principle," Risks, MDPI, vol. 4(4), pages 1-12, December.
    19. Furman, Edward & Wang, Ruodu & Zitikis, Ričardas, 2017. "Gini-type measures of risk and variability: Gini shortfall, capital allocations, and heavy-tailed risks," Journal of Banking & Finance, Elsevier, vol. 83(C), pages 70-84.
    20. Nicole Bäuerle & Ulrich Rieder, 2014. "More Risk-Sensitive Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 39(1), pages 105-120, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:mathme:v:94:y:2021:i:1:d:10.1007_s00186-021-00746-w. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.