IDEAS home Printed from https://ideas.repec.org/a/spr/mathme/v88y2018i2d10.1007_s00186-018-0633-5.html
   My bibliography  Save this article

Risk measurement and risk-averse control of partially observable discrete-time Markov systems

Author

Listed:
  • Jingnan Fan

    (Rutgers University)

  • Andrzej Ruszczyński

    (Rutgers University)

Abstract

We consider risk measurement in controlled partially observable Markov processes in discrete time. We introduce a new concept of conditional stochastic time consistency and we derive the structure of risk measures enjoying this property. We prove that they can be represented by a collection of static law invariant risk measures on the space of function of the observable part of the state. We also derive the corresponding dynamic programming equations. Finally we illustrate the results on a machine deterioration problem.

Suggested Citation

  • Jingnan Fan & Andrzej Ruszczyński, 2018. "Risk measurement and risk-averse control of partially observable discrete-time Markov systems," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 88(2), pages 161-184, October.
    • Handle: RePEc:spr:mathme:v:88:y:2018:i:2:d:10.1007_s00186-018-0633-5
    DOI: 10.1007/s00186-018-0633-5
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00186-018-0633-5
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00186-018-0633-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jingnan Fan & Andrzej Ruszczynski, 2014. "Process-Based Risk Measures and Risk-Averse Control of Discrete-Time Systems," Papers 1411.2675, arXiv.org, revised Nov 2016.
    2. Ogryczak, Wlodzimierz & Ruszczynski, Andrzej, 1999. "From stochastic dominance to mean-risk models: Semideviations as risk measures," European Journal of Operational Research, Elsevier, vol. 116(1), pages 33-50, July.
    3. Alessandro Arlotto & Noah Gans & J. Michael Steele, 2014. "Markov Decision Problems Where Means Bound Variances," Operations Research, INFORMS, vol. 62(4), pages 864-875, August.
    4. ,, 1998. "Problems And Solutions," Econometric Theory, Cambridge University Press, vol. 14(1), pages 151-159, February.
    5. Riedel, Frank, 2004. "Dynamic coherent risk measures," Stochastic Processes and their Applications, Elsevier, vol. 112(2), pages 185-200, August.
    6. A. Jobert & L. C. G. Rogers, 2008. "Valuations And Dynamic Convex Risk Measures," Mathematical Finance, Wiley Blackwell, vol. 18(1), pages 1-22, January.
    7. Runggaldier, Wolfgang J., 1998. "Concepts and methods for discrete and continuous time control under uncertainty," Insurance: Mathematics and Economics, Elsevier, vol. 22(1), pages 25-39, May.
    8. Mannor, Shie & Tsitsiklis, John N., 2013. "Algorithmic aspects of mean–variance optimization in Markov decision processes," European Journal of Operational Research, Elsevier, vol. 231(3), pages 645-653.
    9. Nicole Bäauerle & Ulrich Rieder, 2017. "Partially Observable Risk-Sensitive Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 42(4), pages 1180-1196, November.
    10. Susanne Klöppel & Martin Schweizer, 2007. "Dynamic Indifference Valuation Via Convex Risk Measures," Mathematical Finance, Wiley Blackwell, vol. 17(4), pages 599-627, October.
    11. Berend Roorda & J. M. Schumacher & Jacob Engwerda, 2005. "Coherent Acceptability Measures In Multiperiod Models," Mathematical Finance, Wiley Blackwell, vol. 15(4), pages 589-612, October.
    12. Philippe Artzner & Freddy Delbaen & Jean-Marc Eber & David Heath & Hyejin Ku, 2007. "Coherent multiperiod risk adjusted values and Bellman’s principle," Annals of Operations Research, Springer, vol. 152(1), pages 5-22, July.
    13. ,, 1998. "Problems And Solutions," Econometric Theory, Cambridge University Press, vol. 14(5), pages 687-698, October.
    14. Patrick Cheridito & Michael Kupper, 2011. "Composition Of Time-Consistent Dynamic Monetary Risk Measures In Discrete Time," International Journal of Theoretical and Applied Finance (IJTAF), World Scientific Publishing Co. Pte. Ltd., vol. 14(01), pages 137-162.
    15. ,, 1998. "Problems And Solutions," Econometric Theory, Cambridge University Press, vol. 14(3), pages 381-386, June.
    16. ,, 1998. "Problems And Solutions," Econometric Theory, Cambridge University Press, vol. 14(4), pages 525-537, August.
    17. Eugene A. Feinberg & Pavlo O. Kasyanov & Michael Z. Zgurovsky, 2016. "Partially Observable Total-Cost Markov Decision Processes with Weakly Continuous Transition Probabilities," Mathematics of Operations Research, INFORMS, vol. 41(2), pages 656-681, May.
    18. Jerzy A. Filar & L. C. M. Kallenberg & Huey-Miin Lee, 1989. "Variance-Penalized Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 14(1), pages 147-161, February.
    19. ,, 1998. "Problems And Solutions," Econometric Theory, Cambridge University Press, vol. 14(2), pages 285-292, April.
    20. A. Jaśkiewicz & J. Matkowski & A. S. Nowak, 2013. "Persistently Optimal Policies in Stochastic Dynamic Programming with Generalized Discounting," Mathematics of Operations Research, INFORMS, vol. 38(1), pages 108-121, February.
    21. Chen, Zhiping & Li, Gang & Zhao, Yonggan, 2014. "Time-consistent investment policies in Markovian markets: A case of mean–variance analysis," Journal of Economic Dynamics and Control, Elsevier, vol. 40(C), pages 293-316.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Tomasz R. Bielecki & Igor Cialenco & Andrzej Ruszczy'nski, 2022. "Risk Filtering and Risk-Averse Control of Markovian Systems Subject to Model Uncertainty," Papers 2206.09235, arXiv.org.
    2. Randall Martyr & John Moriarty & Magnus Perninge, 2019. "Discrete-time risk-aware optimal switching with non-adapted costs," Papers 1910.04047, arXiv.org, revised Sep 2021.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jingnan Fan & Andrzej Ruszczynski, 2014. "Process-Based Risk Measures and Risk-Averse Control of Discrete-Time Systems," Papers 1411.2675, arXiv.org, revised Nov 2016.
    2. Dassios, Ioannis K. & Szajowski, Krzysztof J., 2016. "Bayesian optimal control for a non-autonomous stochastic discrete time system," Applied Mathematics and Computation, Elsevier, vol. 274(C), pages 556-564.
    3. Dolf Talman & Zaifu Yang, 2012. "On a Parameterized System of Nonlinear Equations with Economic Applications," Journal of Optimization Theory and Applications, Springer, vol. 154(2), pages 644-671, August.
    4. Zhiqiang Zheng & Balaji Padmanabhan & Steven O. Kimbrough, 2003. "On the Existence and Significance of Data Preprocessing Biases in Web-Usage Mining," INFORMS Journal on Computing, INFORMS, vol. 15(2), pages 148-170, May.
    5. Herings, P.J.J. & Talman, A.J.J. & Yang, Z.F., 1999. "Variational Inequality Problems With a Continuum of Solutions : Existence and Computation," Other publications TiSEM 73e2f01b-ad4d-4447-95ba-a, Tilburg University, School of Economics and Management.
    6. Carlos R. Handy & Daniel Vrinceanu & Carl B. Marth & Harold A. Brooks, 2015. "Pointwise Reconstruction of Wave Functions from Their Moments through Weighted Polynomial Expansions: An Alternative Global-Local Quantization Procedure," Mathematics, MDPI, vol. 3(4), pages 1-24, November.
    7. Allen C. Goodman & Miron Stano, 2000. "Hmos and Health Externalities: A Local Public Good Perspective," Public Finance Review, , vol. 28(3), pages 247-269, May.
    8. Bode, Sven & Michaelowa, Axel, 2003. "Avoiding perverse effects of baseline and investment additionality determination in the case of renewable energy projects," Energy Policy, Elsevier, vol. 31(6), pages 505-517, May.
    9. Ala, Guido & Fasshauer, Gregory E. & Francomano, Elisa & Ganci, Salvatore & McCourt, Michael J., 2017. "An augmented MFS approach for brain activity reconstruction," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 141(C), pages 3-15.
    10. Bettina Campedelli & Andrea Guerrina & Giulia Romano & Chiara Leardini, 2014. "La performance della rete ospedaliera pubblica della regione Veneto. L?impatto delle variabili ambientali e operative sull?efficienza," MECOSAN, FrancoAngeli Editore, vol. 2014(92), pages 119-142.
    11. Haider A. Khan, 2004. "General Conclusions: From Crisis to a Global Political Economy of Freedom," Palgrave Macmillan Books, in: Global Markets and Financial Crises in Asia, chapter 9, pages 193-211, Palgrave Macmillan.
    12. Penn Loh & Zoë Ackerman & Joceline Fidalgo & Rebecca Tumposky, 2022. "Co-Education/Co-Research Partnership: A Critical Approach to Co-Learning between Dudley Street Neighborhood Initiative and Tufts University," Social Sciences, MDPI, vol. 11(2), pages 1-17, February.
    13. Broekhuis, Manda & Vos, Janita F.J., 2003. "Improving organizational sustainability using a quality perspective," Research Report 03A43, University of Groningen, Research Institute SOM (Systems, Organisations and Management).
    14. O'Brien, Raymond & Patacchini, Eleonora, 2003. "Testing the exogeneity assumption in panel data models with "non classical" disturbances," Discussion Paper Series In Economics And Econometrics 0302, Economics Division, School of Social Sciences, University of Southampton.
    15. van der Laan, G. & Talman, A.J.J. & Yang, Z.F., 2002. "Perfection and Stability of Stationary Points with Applications in Noncooperative Games," Discussion Paper 2002-108, Tilburg University, Center for Economic Research.
    16. Edcarlos D. Silva & J. C. Albuquerque & T. R. Cavalcante, 2021. "Fourth-order nonlocal type elliptic problems with indefinite nonlinearities," Partial Differential Equations and Applications, Springer, vol. 2(2), pages 1-22, April.
    17. YongSeog Kim & W. Nick Street & Gary J. Russell & Filippo Menczer, 2005. "Customer Targeting: A Neural Network Approach Guided by Genetic Algorithms," Management Science, INFORMS, vol. 51(2), pages 264-276, February.
    18. Montijano, J.I. & Rández, L. & Van Daele, M. & Calvo, M., 2020. "On the numerical stability of the exponentially fitted methods for first order IVPs," Applied Mathematics and Computation, Elsevier, vol. 379(C).
    19. Yanling Li & Zita Oravecz & Shuai Zhou & Yosef Bodovski & Ian J. Barnett & Guangqing Chi & Yuan Zhou & Naomi P. Friedman & Scott I. Vrieze & Sy-Miin Chow, 2022. "Bayesian Forecasting with a Regime-Switching Zero-Inflated Multilevel Poisson Regression Model: An Application to Adolescent Alcohol Use with Spatial Covariates," Psychometrika, Springer;The Psychometric Society, vol. 87(2), pages 376-402, June.
    20. Jensen, Nathan M. & Li, Quan & Rahman, Aminur, 2007. "Heard melodies are sweet, but those unheard are sweeter : understanding corruption using cross-national firm-level surveys," Policy Research Working Paper Series 4413, The World Bank.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:mathme:v:88:y:2018:i:2:d:10.1007_s00186-018-0633-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.