IDEAS home Printed from https://ideas.repec.org/a/inm/oropre/v62y2014i2p401-417.html
   My bibliography  Save this article

Computational Methods for Risk-Averse Undiscounted Transient Markov Models

Author

Listed:
  • Özlem Çavuş

    (Department of Industrial Engineering, Bilkent University, Ankara 06800, Turkey)

  • Andrzej Ruszczyński

    (Department of Management Science and Information Systems, Rutgers University, Piscataway, New Jersey 08854)

Abstract

The total cost problem for discrete-time controlled transient Markov models is considered. The objective functional is a Markov dynamic risk measure of the total cost. Two solution methods, value and policy iteration, are proposed, and their convergence is analyzed. In the policy iteration method, we propose two algorithms for policy evaluation: the nonsmooth Newton method and convex programming, and we prove their convergence. The results are illustrated on a credit limit control problem.

Suggested Citation

  • Özlem Çavuş & Andrzej Ruszczyński, 2014. "Computational Methods for Risk-Averse Undiscounted Transient Markov Models," Operations Research, INFORMS, vol. 62(2), pages 401-417, April.
    • Handle: RePEc:inm:oropre:v:62:y:2014:i:2:p:401-417
    DOI: 10.1287/opre.2013.1251
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/opre.2013.1251
    Download Restriction: no
    File URL: https://libkey.io/10.1287/opre.2013.1251?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Riedel, Frank, 2004. "Dynamic coherent risk measures," Stochastic Processes and their Applications, Elsevier, vol. 112(2), pages 185-200, August.
    2. Stratton C. Jaquette, 1976. "A Utility Criterion for Markov Decision Processes," Management Science, INFORMS, vol. 23(1), pages 43-49, September.
    3. Dimitri P. Bertsekas & John N. Tsitsiklis, 1991. "An Analysis of Stochastic Shortest Path Problems," Mathematics of Operations Research, INFORMS, vol. 16(3), pages 580-595, August.
    4. So, Meko M.C. & Thomas, Lyn C., 2011. "Modelling the profitability of credit cards by Markov decision processes," European Journal of Operational Research, Elsevier, vol. 212(1), pages 123-130, July.
    5. Philippe Artzner & Freddy Delbaen & Jean‐Marc Eber & David Heath, 1999. "Coherent Measures of Risk," Mathematical Finance, Wiley Blackwell, vol. 9(3), pages 203-228, July.
    6. Ronald A. Howard & James E. Matheson, 1972. "Risk-Sensitive Markov Decision Processes," Management Science, INFORMS, vol. 18(7), pages 356-369, March.
    7. Yoshio Ohtsubo, 2003. "Minimizing risk models in stochastic shortest path problems," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 57(1), pages 79-88, April.
    8. Kang Boda & Jerzy Filar, 2006. "Time Consistent Dynamic Risk Measures," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 63(1), pages 169-186, February.
    9. Jonathan Eckstein & Steven T. Riedmueller, 2002. "YASAI: Yet Another Add-in for Teaching Elementary Monte Carlo Simulation in Excel," INFORMS Transactions on Education, INFORMS, vol. 2(2), pages 12-26, January.
    10. Ogryczak, Wlodzimierz & Ruszczynski, Andrzej, 1999. "From stochastic dominance to mean-risk models: Semideviations as risk measures," European Journal of Operational Research, Elsevier, vol. 116(1), pages 33-50, July.
    11. Rolando Cavazos-Cadena & Emmanuel Fernández-Gaucherand, 1999. "Controlled Markov chains with risk-sensitive criteria: Average cost, optimality equations, and optimal solutions," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 49(2), pages 299-324, April.
    12. Jerzy A. Filar & L. C. M. Kallenberg & Huey-Miin Lee, 1989. "Variance-Penalized Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 14(1), pages 147-161, February.
    13. Rockafellar, R. Tyrrell & Uryasev, Stanislav, 2002. "Conditional value-at-risk for general loss distributions," Journal of Banking & Finance, Elsevier, vol. 26(7), pages 1443-1471, July.
    14. Nie, Yu (Marco) & Wu, Xing, 2009. "Shortest path problem considering on-time arrival probability," Transportation Research Part B: Methodological, Elsevier, vol. 43(6), pages 597-613, July.
    15. Georg Ch Pflug & Werner Römisch, 2007. "Modeling, Measuring and Managing Risk," World Scientific Books, World Scientific Publishing Co. Pte. Ltd., number 6478, December.
    16. Tomasz Bielecki & Daniel Hernández-Hernández & Stanley R. Pliska, 1999. "Risk sensitive control of finite state Markov chains in discrete time, with applications to portfolio management," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 50(2), pages 167-188, October.
    17. Philippe Artzner & Freddy Delbaen & Jean-Marc Eber & David Heath & Hyejin Ku, 2007. "Coherent multiperiod risk adjusted values and Bellman’s principle," Annals of Operations Research, Springer, vol. 152(1), pages 5-22, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Malekipirbazari, Milad & Çavuş, Özlem, 2024. "Index policy for multiarmed bandit problem with dynamic risk measures," European Journal of Operational Research, Elsevier, vol. 312(2), pages 627-640.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sıtkı Gülten & Andrzej Ruszczyński, 2015. "Two-stage portfolio optimization with higher-order conditional measures of risk," Annals of Operations Research, Springer, vol. 229(1), pages 409-427, June.
    2. Dan A. Iancu & Marek Petrik & Dharmashankar Subramanian, 2015. "Tight Approximations of Dynamic Risk Measures," Mathematics of Operations Research, INFORMS, vol. 40(3), pages 655-682, March.
    3. Cui, Xiangyu & Gao, Jianjun & Shi, Yun & Zhu, Shushang, 2019. "Time-consistent and self-coordination strategies for multi-period mean-Conditional Value-at-Risk portfolio selection," European Journal of Operational Research, Elsevier, vol. 276(2), pages 781-789.
    4. Daniel R. Jiang & Warren B. Powell, 2018. "Risk-Averse Approximate Dynamic Programming with Quantile-Based Risk Measures," Mathematics of Operations Research, INFORMS, vol. 43(2), pages 554-579, May.
    5. Prékopa, András & Lee, Jinwook, 2018. "Risk tomography," European Journal of Operational Research, Elsevier, vol. 265(1), pages 149-168.
    6. Jingnan Fan & Andrzej Ruszczyński, 2018. "Risk measurement and risk-averse control of partially observable discrete-time Markov systems," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 88(2), pages 161-184, October.
    7. Jonathan Eckstein & Deniz Eskandani & Jingnan Fan, 2016. "Multilevel Optimization Modeling for Risk-Averse Stochastic Programming," INFORMS Journal on Computing, INFORMS, vol. 28(1), pages 112-128, February.
    8. Naomi Miller & Andrzej Ruszczyński, 2011. "Risk-Averse Two-Stage Stochastic Linear Programming: Modeling and Decomposition," Operations Research, INFORMS, vol. 59(1), pages 125-132, February.
    9. Andreas H Hamel, 2018. "Monetary Measures of Risk," Papers 1812.04354, arXiv.org.
    10. Mahmutoğulları, Ali İrfan & Çavuş, Özlem & Aktürk, M. Selim, 2018. "Bounds on risk-averse mixed-integer multi-stage stochastic programming problems with mean-CVaR," European Journal of Operational Research, Elsevier, vol. 266(2), pages 595-608.
    11. Christopher W. Miller & Insoon Yang, 2015. "Optimal Control of Conditional Value-at-Risk in Continuous Time," Papers 1512.05015, arXiv.org, revised Jan 2017.
    12. Zsolt Bihary & Péter Csóka & Dávid Zoltán Szabó, 2020. "Spectral risk measure of holding stocks in the long run," Annals of Operations Research, Springer, vol. 295(1), pages 75-89, December.
    13. Roberto Cominetti & Alfredo Torrico, 2016. "Additive Consistency of Risk Measures and Its Application to Risk-Averse Routing in Networks," Mathematics of Operations Research, INFORMS, vol. 41(4), pages 1510-1521, November.
    14. Ricardo Collado & Dávid Papp & Andrzej Ruszczyński, 2012. "Scenario decomposition of risk-averse multistage stochastic programming problems," Annals of Operations Research, Springer, vol. 200(1), pages 147-170, November.
    15. Malekipirbazari, Milad & Çavuş, Özlem, 2024. "Index policy for multiarmed bandit problem with dynamic risk measures," European Journal of Operational Research, Elsevier, vol. 312(2), pages 627-640.
    16. Karel Sladký, 2013. "Risk-Sensitive and Mean Variance Optimality in Markov Decision Processes," Czech Economic Review, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, vol. 7(3), pages 146-161, November.
    17. Ji, Ronglin & Shi, Xuejun & Wang, Shijie & Zhou, Jinming, 2019. "Dynamic risk measures for processes via backward stochastic differential equations," Insurance: Mathematics and Economics, Elsevier, vol. 86(C), pages 43-50.
    18. Malavasi, Matteo & Ortobelli Lozza, Sergio & Trück, Stefan, 2021. "Second order of stochastic dominance efficiency vs mean variance efficiency," European Journal of Operational Research, Elsevier, vol. 290(3), pages 1192-1206.
    19. Alois Pichler, 2013. "Premiums And Reserves, Adjusted By Distortions," Papers 1304.0490, arXiv.org.
    20. Zachary Feinstein & Birgit Rudloff, 2018. "Scalar multivariate risk measures with a single eligible asset," Papers 1807.10694, arXiv.org, revised Feb 2021.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:oropre:v:62:y:2014:i:2:p:401-417. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.