IDEAS home Printed from https://ideas.repec.org/a/spr/mathme/v74y2011i3p361-379.html
   My bibliography  Save this article

Markov Decision Processes with Average-Value-at-Risk criteria

Author

Listed:
  • Nicole Bäuerle
  • Jonathan Ott

Abstract

We investigate the problem of minimizing the Average-Value-at-Risk (AVaR τ ) of the discounted cost over a finite and an infinite horizon which is generated by a Markov Decision Process (MDP). We show that this problem can be reduced to an ordinary MDP with extended state space and give conditions under which an optimal policy exists. We also give a time-consistent interpretation of the AVaR τ . At the end we consider a numerical example which is a simple repeated casino game. It is used to discuss the influence of the risk aversion parameter τ of the AVaR τ -criterion. Copyright Springer-Verlag 2011

Suggested Citation

  • Nicole Bäuerle & Jonathan Ott, 2011. "Markov Decision Processes with Average-Value-at-Risk criteria," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 74(3), pages 361-379, December.
    • Handle: RePEc:spr:mathme:v:74:y:2011:i:3:p:361-379
    DOI: 10.1007/s00186-011-0367-0
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s00186-011-0367-0
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s00186-011-0367-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jocelyne Bion-Nadal, 2008. "Dynamic risk measures: Time consistency and risk measures from BMO martingales," Finance and Stochastics, Springer, vol. 12(2), pages 219-244, April.
    2. Acerbi, Carlo & Tasche, Dirk, 2002. "On the coherence of expected shortfall," Journal of Banking & Finance, Elsevier, vol. 26(7), pages 1487-1503, July.
    3. Duan Li & Wan‐Lung Ng, 2000. "Optimal Dynamic Portfolio Selection: Multiperiod Mean‐Variance Formulation," Mathematical Finance, Wiley Blackwell, vol. 10(3), pages 387-406, July.
    4. Kang Boda & Jerzy Filar, 2006. "Time Consistent Dynamic Risk Measures," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 63(1), pages 169-186, February.
    5. Ronald A. Howard & James E. Matheson, 1972. "Risk-Sensitive Markov Decision Processes," Management Science, INFORMS, vol. 18(7), pages 356-369, March.
    6. Rockafellar, R. Tyrrell & Uryasev, Stanislav, 2002. "Conditional value-at-risk for general loss distributions," Journal of Banking & Finance, Elsevier, vol. 26(7), pages 1443-1471, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nicole Bäuerle & Alexander Glauner, 2021. "Minimizing spectral risk measures applied to Markov decision processes," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 94(1), pages 35-69, August.
    2. Charilaos Mertzanis, 2013. "Risk Management Challenges after the Financial Crisis," Economic Notes, Banca Monte dei Paschi di Siena SpA, vol. 42(3), pages 285-320, November.
    3. Bäuerle, Nicole & Glauner, Alexander, 2022. "Markov decision processes with recursive risk measures," European Journal of Operational Research, Elsevier, vol. 296(3), pages 953-966.
    4. Christopher W. Miller & Insoon Yang, 2015. "Optimal Control of Conditional Value-at-Risk in Continuous Time," Papers 1512.05015, arXiv.org, revised Jan 2017.
    5. So, Mee Chi & Thomas, Lyn C. & Huang, Bo, 2016. "Lending decisions with limits on capital available: The polygamous marriage problem," European Journal of Operational Research, Elsevier, vol. 249(2), pages 407-416.
    6. Seungki Min & Ciamac C. Moallemi & Costis Maglaras, 2022. "Risk-Sensitive Optimal Execution via a Conditional Value-at-Risk Objective," Papers 2201.11962, arXiv.org.
    7. Vamsi K. Potluru & Daniel Borrajo & Andrea Coletta & Niccol`o Dalmasso & Yousef El-Laham & Elizabeth Fons & Mohsen Ghassemi & Sriram Gopalakrishnan & Vikesh Gosai & Eleonora Kreav{c}i'c & Ganapathy Ma, 2023. "Synthetic Data Applications in Finance," Papers 2401.00081, arXiv.org, revised Mar 2024.
    8. Nicole Bauerle & Alexander Glauner, 2020. "Minimizing Spectral Risk Measures Applied to Markov Decision Processes," Papers 2012.04521, arXiv.org.
    9. Julio Backhoff Veraguas & A. Max Reppen & Ludovic Tangpi, 2020. "Stochastic control of optimized certainty equivalents," Papers 2001.10108, arXiv.org, revised Jun 2022.
    10. Nicole Bauerle & Alexander Glauner, 2020. "Distributionally Robust Markov Decision Processes and their Connection to Risk Measures," Papers 2007.13103, arXiv.org.
    11. Nicole Bäuerle & Ulrich Rieder, 2014. "More Risk-Sensitive Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 39(1), pages 105-120, February.
    12. Constantin Waubert de Puiseau & Richard Meyes & Tobias Meisen, 2022. "On reliability of reinforcement learning based production scheduling systems: a comparative survey," Journal of Intelligent Manufacturing, Springer, vol. 33(4), pages 911-927, April.
    13. Qiuli Liu & Wai-Ki Ching & Xianping Guo, 2023. "Zero-sum stochastic games with the average-value-at-risk criterion," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 31(3), pages 618-647, October.
    14. Juri Hinz & Tanya Tarnopolskaya & Jeremy Yee, 2020. "Efficient algorithms of pathwise dynamic programming for decision optimization in mining operations," Annals of Operations Research, Springer, vol. 286(1), pages 583-615, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gao, Jianjun & Xiong, Yan & Li, Duan, 2016. "Dynamic mean-risk portfolio selection with multiple risk measures in continuous-time," European Journal of Operational Research, Elsevier, vol. 249(2), pages 647-656.
    2. Zhiping Chen & Jia Liu & Gang Li & Zhe Yan, 2016. "Composite time-consistent multi-period risk measure and its application in optimal portfolio selection," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 24(3), pages 515-540, October.
    3. Strub, Moris S. & Li, Duan & Cui, Xiangyu & Gao, Jianjun, 2019. "Discrete-time mean-CVaR portfolio selection and time-consistency induced term structure of the CVaR," Journal of Economic Dynamics and Control, Elsevier, vol. 108(C).
    4. Özlem Çavuş & Andrzej Ruszczyński, 2014. "Computational Methods for Risk-Averse Undiscounted Transient Markov Models," Operations Research, INFORMS, vol. 62(2), pages 401-417, April.
    5. Xiangyu Cui & Xun Li & Duan Li & Yun Shi, 2014. "Time Consistent Behavior Portfolio Policy for Dynamic Mean-Variance Formulation," Papers 1408.6070, arXiv.org, revised Aug 2015.
    6. Alexander, Gordon J. & Baptista, Alexandre M. & Yan, Shu, 2012. "When more is less: Using multiple constraints to reduce tail risk," Journal of Banking & Finance, Elsevier, vol. 36(10), pages 2693-2716.
    7. Kull, Andreas, 2009. "Sharing Risk – An Economic Perspective," ASTIN Bulletin, Cambridge University Press, vol. 39(2), pages 591-613, November.
    8. Gauvin, Charles & Delage, Erick & Gendreau, Michel, 2017. "Decision rule approximations for the risk averse reservoir management problem," European Journal of Operational Research, Elsevier, vol. 261(1), pages 317-336.
    9. Brian Tomlin & Yimin Wang, 2005. "On the Value of Mix Flexibility and Dual Sourcing in Unreliable Newsvendor Networks," Manufacturing & Service Operations Management, INFORMS, vol. 7(1), pages 37-57, June.
    10. Roman, Diana & Mitra, Gautam & Zverovich, Victor, 2013. "Enhanced indexation based on second-order stochastic dominance," European Journal of Operational Research, Elsevier, vol. 228(1), pages 273-281.
    11. Dirk Tasche, 2015. "Fitting a distribution to Value-at-Risk and Expected Shortfall, with an application to covered bonds," Papers 1505.07484, arXiv.org, revised Nov 2015.
    12. Fermanian, Jean-David & Scaillet, Olivier, 2005. "Sensitivity analysis of VaR and Expected Shortfall for portfolios under netting agreements," Journal of Banking & Finance, Elsevier, vol. 29(4), pages 927-958, April.
    13. Maziar Sahamkhadam, 2021. "Dynamic copula-based expectile portfolios," Journal of Asset Management, Palgrave Macmillan, vol. 22(3), pages 209-223, May.
    14. Martin Herdegen & Cosimo Munari, 2023. "An elementary proof of the dual representation of Expected Shortfall," Papers 2306.14506, arXiv.org.
    15. Xiangyu Cui & Duan Li & Xun Li, 2014. "Mean-Variance Policy for Discrete-time Cone Constrained Markets: The Consistency in Efficiency and Minimum-Variance Signed Supermartingale Measure," Papers 1403.0718, arXiv.org.
    16. Jiang, Chun-Fu & Peng, Hong-Yi & Yang, Yu-Kuan, 2016. "Tail variance of portfolio under generalized Laplace distribution," Applied Mathematics and Computation, Elsevier, vol. 282(C), pages 187-203.
    17. Michael B. Gordy & Sandeep Juneja, 2010. "Nested Simulation in Portfolio Risk Measurement," Management Science, INFORMS, vol. 56(10), pages 1833-1848, October.
    18. Rudloff, Birgit & Street, Alexandre & Valladão, Davi M., 2014. "Time consistency and risk averse dynamic decision models: Definition, interpretation and practical consequences," European Journal of Operational Research, Elsevier, vol. 234(3), pages 743-750.
    19. Arismendi, Juan C. & Broda, Simon, 2017. "Multivariate elliptical truncated moments," Journal of Multivariate Analysis, Elsevier, vol. 157(C), pages 29-44.
    20. Karma, Otto & Sander, Priit, 2006. "The impact of financial leverage on risk of equity measured by loss-oriented risk measures: An option pricing approach," European Journal of Operational Research, Elsevier, vol. 175(3), pages 1340-1356, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:mathme:v:74:y:2011:i:3:p:361-379. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.