IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v12y2023i1p38-d1305724.html
   My bibliography  Save this article

Discounted Risk-Sensitive Optimal Control of Switching Diffusions: Viscosity Solution and Numerical Approximation

Author

Listed:
  • Xianggang Lu

    (School of Mathematics and Statistics, Guangdong University of Technology, Guangzhou 510006, China)

  • Lin Sun

    (School of Mathematics and Statistics, Guangdong University of Technology, Guangzhou 510006, China)

Abstract

This work considers the infinite horizon discounted risk-sensitive optimal control problem for the switching diffusions with a compact control space and controlled through the drift; thus, the the generator of the switching diffusions also depends on the controls. Note that the running cost of interest can be unbounded, so a decent estimation on the value function is obtained, under suitable conditions. To solve such a risk-sensitive optimal control problem, we adopt the viscosity solution methods and propose a numerical approximation scheme. We can verify that the value function of the optimal control problem solves the optimality equation as the unique viscosity solution. The optimality equation is also called the Hamilton–Jacobi–Bellman (HJB) equation, which is a second-order partial differential equation (PDE). Since, the explicit solutions to such PDEs are usually difficult to obtain, the finite difference approximation scheme is derived to approximate the value function. As a byproduct, the ϵ -optimal control of finite difference type is also obtained.

Suggested Citation

  • Xianggang Lu & Lin Sun, 2023. "Discounted Risk-Sensitive Optimal Control of Switching Diffusions: Viscosity Solution and Numerical Approximation," Mathematics, MDPI, vol. 12(1), pages 1-24, December.
    • Handle: RePEc:gam:jmathe:v:12:y:2023:i:1:p:38-:d:1305724
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/12/1/38/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/12/1/38/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Nguyen, Dang Hai & Yin, George & Zhu, Chao, 2017. "Certain properties related to well posedness of switching diffusions," Stochastic Processes and their Applications, Elsevier, vol. 127(10), pages 3135-3158.
    2. Xianggang Lu, 2019. "Constrained optimality for controlled switching diffusions with an application to stock purchasing," Quantitative Finance, Taylor & Francis Journals, vol. 19(12), pages 2069-2085, December.
    3. Peter Grandits & Friedrich Hubalek & Walter Schachermayer & Mislav Žigo, 2007. "Optimal expected exponential utility of dividend payments in a Brownian risk model," Scandinavian Actuarial Journal, Taylor & Francis Journals, vol. 2007(2), pages 73-107.
    4. V. S. Borkar, 2002. "Q-Learning for Risk-Sensitive Control," Mathematics of Operations Research, INFORMS, vol. 27(2), pages 294-311, May.
    5. Ronald A. Howard & James E. Matheson, 1972. "Risk-Sensitive Markov Decision Processes," Management Science, INFORMS, vol. 18(7), pages 356-369, March.
    6. Arapostathis, Ari & Biswas, Anup, 2018. "Infinite horizon risk-sensitive control of diffusions without any blanket stability assumptions," Stochastic Processes and their Applications, Elsevier, vol. 128(5), pages 1485-1524.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nicole Bäuerle & Anna Jaśkiewicz, 2024. "Markov decision processes with risk-sensitive criteria: an overview," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 99(1), pages 141-178, April.
    2. Arnab Basu & Tirthankar Bhattacharyya & Vivek S. Borkar, 2008. "A Learning Algorithm for Risk-Sensitive Cost," Mathematics of Operations Research, INFORMS, vol. 33(4), pages 880-898, November.
    3. Basu, Arnab & Ghosh, Mrinal Kanti, 2014. "Zero-sum risk-sensitive stochastic games on a countable state space," Stochastic Processes and their Applications, Elsevier, vol. 124(1), pages 961-983.
    4. Zeynep Erkin & Matthew D. Bailey & Lisa M. Maillart & Andrew J. Schaefer & Mark S. Roberts, 2010. "Eliciting Patients' Revealed Preferences: An Inverse Markov Decision Process Approach," Decision Analysis, INFORMS, vol. 7(4), pages 358-365, December.
    5. Dongping Qi & Adam Dhillon & Alexander Vladimirsky, 2025. "Optimality and Robustness in Path-Planning Under Initial Uncertainty," Dynamic Games and Applications, Springer, vol. 15(2), pages 637-663, May.
    6. Monahan, George E. & Sobel, Matthew J., 1997. "Risk-Sensitive Dynamic Market Share Attraction Games," Games and Economic Behavior, Elsevier, vol. 20(2), pages 149-160, August.
    7. V. S. Borkar & S. P. Meyn, 2002. "Risk-Sensitive Optimal Control for Markov Decision Processes with Monotone Cost," Mathematics of Operations Research, INFORMS, vol. 27(1), pages 192-209, February.
    8. Jelito, Damian & Pitera, Marcin & Stettner, Łukasz, 2021. "Risk sensitive optimal stopping," Stochastic Processes and their Applications, Elsevier, vol. 136(C), pages 125-144.
    9. Erick Delage & Shie Mannor, 2010. "Percentile Optimization for Markov Decision Processes with Parameter Uncertainty," Operations Research, INFORMS, vol. 58(1), pages 203-213, February.
    10. Naci Saldi & Tamer Bas¸ ar & Maxim Raginsky, 2020. "Approximate Markov-Nash Equilibria for Discrete-Time Risk-Sensitive Mean-Field Games," Mathematics of Operations Research, INFORMS, vol. 45(4), pages 1596-1620, November.
    11. Dellaert, N.P. & Frenk, J.B.G. & van Rijsoort, L.P., 1993. "Optimal claim behaviour for vehicle damage insurances," Econometric Institute Research Papers 11669, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    12. Hernandez-Castorena, Gerardo Armando & Alcorta-Garcia, Maria Aracelia & Saenz-Esqueda, Jose Armando & Mendez, Gerardo Maximiliano, 2024. "Lyapunov’s stability analysis for first degree polynomial systems, subject to risk-sensitive control," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 226(C), pages 464-473.
    13. Rolando Cavazos-Cadena, 2009. "Solutions of the average cost optimality equation for finite Markov decision chains: risk-sensitive and risk-neutral criteria," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 70(3), pages 541-566, December.
    14. Rolando Cavazos-Cadena, 2018. "Characterization of the Optimal Risk-Sensitive Average Cost in Denumerable Markov Decision Chains," Mathematics of Operations Research, INFORMS, vol. 43(3), pages 1025-1050, August.
    15. Yanwei Jia, 2024. "Continuous-time Risk-sensitive Reinforcement Learning via Quadratic Variation Penalty," Papers 2404.12598, arXiv.org.
    16. Grzegorz Hałaj, 2016. "Dynamic Balance Sheet Model With Liquidity Risk," International Journal of Theoretical and Applied Finance (IJTAF), World Scientific Publishing Co. Pte. Ltd., vol. 19(07), pages 1-37, November.
    17. Zhou, Baoquan & Shi, Ningzhong, 2024. "Analysis of a stochastic SEIS epidemic model motivated by Black–Karasinski process: Probability density function," Chaos, Solitons & Fractals, Elsevier, vol. 189(P2).
    18. Kumar, Uday M & Bhat, Sanjay P. & Kavitha, Veeraruna & Hemachandra, Nandyala, 2023. "Approximate solutions to constrained risk-sensitive Markov decision processes," European Journal of Operational Research, Elsevier, vol. 310(1), pages 249-267.
    19. Daniel Hernández Hernández & Diego Hernández Bustos, 2017. "Local Poisson Equations Associated with Discrete-Time Markov Control Processes," Journal of Optimization Theory and Applications, Springer, vol. 173(1), pages 1-29, April.
    20. Xianhua Peng & Xiang Zhou & Bo Xiao & Yi Wu, 2024. "A Risk Sensitive Contract-unified Reinforcement Learning Approach for Option Hedging," Papers 2411.09659, arXiv.org.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:12:y:2023:i:1:p:38-:d:1305724. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.