IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2207.00932.html

Deep Bellman Hedging

Author

Listed:
  • Hans Buehler
  • Phillip Murray
  • Ben Wood

Abstract

We present an actor-critic-type reinforcement learning algorithm for solving the problem of hedging a portfolio of financial instruments such as securities and over-the-counter derivatives using purely historic data. The key characteristics of our approach are: the ability to hedge with derivatives such as forwards, swaps, futures, options; incorporation of trading frictions such as trading cost and liquidity constraints; applicability for any reasonable portfolio of financial instruments; realistic, continuous state and action spaces; and formal risk-adjusted return objectives. Most importantly, the trained model provides an optimal hedge for arbitrary initial portfolios and market states without the need for re-training. We also prove existence of finite solutions to our Bellman equation, and show the relation to our vanilla Deep Hedging approach

Suggested Citation

  • Hans Buehler & Phillip Murray & Ben Wood, 2022. "Deep Bellman Hedging," Papers 2207.00932, arXiv.org, revised Jun 2024.
    • Handle: RePEc:arx:papers:2207.00932
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2207.00932
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Kai Detlefsen & Giacomo Scandolo, 2005. "Conditional and dynamic convex risk measures," Finance and Stochastics, Springer, vol. 9(4), pages 539-561, October.
    2. Igor Halperin, 2019. "The QLBS Q-Learner goes NuQLear: fitted Q iteration, inverse RL, and option portfolios," Quantitative Finance, Taylor & Francis Journals, vol. 19(9), pages 1543-1553, September.
    3. repec:hum:wpaper:sfb649dp2005-006 is not listed on IDEAS
    4. Aharon Ben‐Tal & Marc Teboulle, 2007. "An Old‐New Concept Of Convex Risk Measures: The Optimized Certainty Equivalent," Mathematical Finance, Wiley Blackwell, vol. 17(3), pages 449-476, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Qinyu Wu & Fan Yang & Ping Zhang, 2023. "Conditional generalized quantiles based on expected utility model and equivalent characterization of properties," Papers 2301.12420, arXiv.org.
    2. Fei Sun & Jingchao Li & Jieming Zhou, 2018. "Dynamic risk measures for fluctuations in market volatility under Bochner-Lebesgue spaces," Papers 1806.01166, arXiv.org, revised Jan 2026.
    3. Geissel Sebastian & Sass Jörn & Seifried Frank Thomas, 2018. "Optimal expected utility risk measures," Statistics & Risk Modeling, De Gruyter, vol. 35(1-2), pages 73-87, January.
    4. Dan A. Iancu & Marek Petrik & Dharmashankar Subramanian, 2015. "Tight Approximations of Dynamic Risk Measures," Mathematics of Operations Research, INFORMS, vol. 40(3), pages 655-682, March.
    5. Giulio Principi & Fabio Maccheroni, 2022. "Conditional divergence risk measures," Papers 2211.04592, arXiv.org.
    6. Babacar Seck & Robert J. Elliott, 2021. "Regime Switching Entropic Risk Measures on Crude Oil Pricing," Papers 2112.13041, arXiv.org.
    7. Andreas H Hamel, 2018. "Monetary Measures of Risk," Papers 1812.04354, arXiv.org.
    8. Tomasz R. Bielecki & Igor Cialenco & Marcin Pitera, 2014. "A unified approach to time consistency of dynamic risk measures and dynamic performance measures in discrete time," Papers 1409.7028, arXiv.org, revised Sep 2017.
    9. Eskandarzadeh, Saman & Eshghi, Kourosh, 2013. "Decision tree analysis for a risk averse decision maker: CVaR Criterion," European Journal of Operational Research, Elsevier, vol. 231(1), pages 131-140.
    10. Davi Michel Valladão & Álvaro Veiga & Alexandre Street, 2018. "A Linear Stochastic Programming Model for Optimal Leveraged Portfolio Selection," Computational Economics, Springer;Society for Computational Economics, vol. 51(4), pages 1021-1032, April.
    11. Lacker Daniel, 2018. "Law invariant risk measures and information divergences," Dependence Modeling, De Gruyter, vol. 6(1), pages 228-258, November.
    12. Oscar Dowson & David P. Morton & Bernardo K. Pagnoncelli, 2025. "Incorporating convex risk measures into multistage stochastic programming algorithms," Annals of Operations Research, Springer, vol. 348(2), pages 807-831, May.
    13. Ji, Ronglin & Shi, Xuejun & Wang, Shijie & Zhou, Jinming, 2019. "Dynamic risk measures for processes via backward stochastic differential equations," Insurance: Mathematics and Economics, Elsevier, vol. 86(C), pages 43-50.
    14. Zachary Feinstein & Birgit Rudloff, 2018. "Scalar multivariate risk measures with a single eligible asset," Papers 1807.10694, arXiv.org, revised Feb 2021.
    15. Pascal Franc{c}ois & Genevi`eve Gauthier & Fr'ed'eric Godin & Carlos Octavio P'erez Mendoza, 2024. "Enhancing Deep Hedging of Options with Implied Volatility Surface Feedback Information," Papers 2407.21138, arXiv.org, revised Aug 2025.
    16. Guanyu Jin & Roger J. A. Laeven & Dick den Hertog & Aharon Ben-Tal, 2024. "Constructing Uncertainty Sets for Robust Risk Measures: A Composition of $\phi$-Divergences Approach to Combat Tail Uncertainty," Papers 2412.05234, arXiv.org.
    17. Alessandro Doldi & Marco Frittelli, 2023. "Entropy martingale optimal transport and nonlinear pricing–hedging duality," Finance and Stochastics, Springer, vol. 27(2), pages 255-304, April.
    18. Yuanying Guan & Zhanyi Jiao & Ruodu Wang, 2022. "A reverse ES (CVaR) optimization formula," Papers 2203.02599, arXiv.org, revised May 2023.
    19. Zhang, Yu & Wang, Yu & Tang, Jiafu & Lim, Andrew, 2020. "Mitigating overtime risk in tactical surgical scheduling," Omega, Elsevier, vol. 93(C).
    20. Jinwook Lee & András Prékopa, 2015. "Decision-making from a risk assessment perspective for Corporate Mergers and Acquisitions," Computational Management Science, Springer, vol. 12(2), pages 243-266, April.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2207.00932. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.