IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2401.08600.html
   My bibliography  Save this paper

Reinforcement Learning and Deep Stochastic Optimal Control for Final Quadratic Hedging

Author

Listed:
  • Bernhard Hientzsch

Abstract

We consider two data driven approaches, Reinforcement Learning (RL) and Deep Trajectory-based Stochastic Optimal Control (DTSOC) for hedging a European call option without and with transaction cost according to a quadratic hedging P&L objective at maturity ("variance-optimal hedging" or "final quadratic hedging"). We study the performance of the two approaches under various market environments (modeled via the Black-Scholes and/or the log-normal SABR model) to understand their advantages and limitations. Without transaction costs and in the Black-Scholes model, both approaches match the performance of the variance-optimal Delta hedge. In the log-normal SABR model without transaction costs, they match the performance of the variance-optimal Barlett's Delta hedge. Agents trained on Black-Scholes trajectories with matching initial volatility but used on SABR trajectories match the performance of Bartlett's Delta hedge in average cost, but show substantially wider variance. To apply RL approaches to these problems, P&L at maturity is written as sum of step-wise contributions and variants of RL algorithms are implemented and used that minimize expectation of second moments of such sums.

Suggested Citation

  • Bernhard Hientzsch, 2023. "Reinforcement Learning and Deep Stochastic Optimal Control for Final Quadratic Hedging," Papers 2401.08600, arXiv.org.
    • Handle: RePEc:arx:papers:2401.08600
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2401.08600
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jian Liang & Zhe Xu & Peter Li, 2021. "Deep learning-based least squares forward-backward stochastic differential equation solver for high-dimensional derivative pricing," Quantitative Finance, Taylor & Francis Journals, vol. 21(8), pages 1309-1323, August.
    2. Nicolas Boursin & Carl Remlinger & Joseph Mikael & Carol Anne Hargreaves, 2022. "Deep Generators on Commodity Markets; application to Deep Hedging," Papers 2205.13942, arXiv.org.
    3. Bernhard Hientzsch, 2019. "Introduction to Solving Quant Finance Problems with Time-Stepped FBSDE and Deep Learning," Papers 1911.12231, arXiv.org.
    4. Nicolas Boursin & Carl Remlinger & Joseph Mikael, 2022. "Deep Generators on Commodity Markets Application to Deep Hedging," Risks, MDPI, vol. 11(1), pages 1-18, December.
    5. Arun Kumar Polala & Bernhard Hientzsch, 2023. "Parametric Differential Machine Learning for Pricing and Calibration," Papers 2302.06682, arXiv.org, revised Feb 2023.
    6. Martin Keller-Ressel, 2022. "Bartlett's Delta revisited: Variance-optimal hedging in the lognormal SABR and in the rough Bergomi model," Papers 2207.13573, arXiv.org.
    7. Yajie Yu & Narayan Ganesan & Bernhard Hientzsch, 2023. "Backward Deep BSDE Methods and Applications to Nonlinear Problems," Risks, MDPI, vol. 11(3), pages 1-16, March.
    8. Narayan Ganesan & Yajie Yu & Bernhard Hientzsch, 2020. "Pricing Barrier Options with DeepBSDEs," Papers 2005.10966, arXiv.org.
    9. Patrick S. Hagan & Andrew Lesniewski, 2017. "Bartlett's delta in the SABR model," Papers 1704.03110, arXiv.org, revised May 2020.
    10. A. Max Reppen & H. Mete Soner & Valentin Tissot-Daguette, 2022. "Deep Stochastic Optimization in Finance," Papers 2205.04604, arXiv.org.
    11. Samuel N. Cohen & Christoph Reisinger & Sheng Wang, 2021. "Arbitrage-free neural-SDE market models," Papers 2105.11053, arXiv.org, revised Aug 2021.
    12. Jay Cao & Jacky Chen & John Hull & Zissis Poulos, 2021. "Deep Hedging of Derivatives Using Reinforcement Learning," Papers 2103.16409, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ali Fathi & Bernhard Hientzsch, 2023. "A Comparison of Reinforcement Learning and Deep Trajectory Based Stochastic Control Agents for Stepwise Mean-Variance Hedging," Papers 2302.07996, arXiv.org, revised Nov 2023.
    2. Yajie Yu & Bernhard Hientzsch & Narayan Ganesan, 2020. "Backward Deep BSDE Methods and Applications to Nonlinear Problems," Papers 2006.07635, arXiv.org.
    3. Vedant Choudhary & Sebastian Jaimungal & Maxime Bergeron, 2023. "FuNVol: A Multi-Asset Implied Volatility Market Simulator using Functional Principal Components and Neural SDEs," Papers 2303.00859, arXiv.org, revised Dec 2023.
    4. Alexandre Carbonneau & Fr'ed'eric Godin, 2021. "Deep equal risk pricing of financial derivatives with non-translation invariant risk measures," Papers 2107.11340, arXiv.org.
    5. Federico Giorgi & Stefano Herzel & Paolo Pigato, 2023. "A Reinforcement Learning Algorithm for Trading Commodities," CEIS Research Paper 552, Tor Vergata University, CEIS, revised 18 Feb 2023.
    6. Jay Cao & Jacky Chen & Soroush Farghadani & John Hull & Zissis Poulos & Zeyu Wang & Jun Yuan, 2022. "Gamma and Vega Hedging Using Deep Distributional Reinforcement Learning," Papers 2205.05614, arXiv.org, revised Jan 2023.
    7. Rong Du & Duy-Minh Dang, 2023. "Fourier Neural Network Approximation of Transition Densities in Finance," Papers 2309.03966, arXiv.org.
    8. Ben Hambly & Renyuan Xu & Huining Yang, 2021. "Recent Advances in Reinforcement Learning in Finance," Papers 2112.04553, arXiv.org, revised Feb 2023.
    9. Lorenc Kapllani & Long Teng, 2024. "A backward differential deep learning-based algorithm for solving high-dimensional nonlinear backward stochastic differential equations," Papers 2404.08456, arXiv.org.
    10. Pierre Renucci, 2023. "Optimal Linear Signal: An Unsupervised Machine Learning Framework to Optimize PnL with Linear Signals," Papers 2401.05337, arXiv.org.
    11. Yannick Limmer & Blanka Horvath, 2023. "Robust Hedging GANs," Papers 2307.02310, arXiv.org.
    12. Josef Teichmann & Hanna Wutte, 2023. "Machine Learning-powered Pricing of the Multidimensional Passport Option," Papers 2307.14887, arXiv.org.
    13. Christa Cuchiero & Philipp Schmocker & Josef Teichmann, 2023. "Global universal approximation of functional input maps on weighted spaces," Papers 2306.03303, arXiv.org, revised Feb 2024.
    14. Beatrice Acciaio & Anastasis Kratsios & Gudmund Pammer, 2022. "Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformer," Papers 2201.13094, arXiv.org, revised Mar 2023.
    15. Ariel Neufeld & Julian Sester & Mario v{S}iki'c, 2022. "Markov Decision Processes under Model Uncertainty," Papers 2206.06109, arXiv.org, revised Jan 2023.
    16. Wing Fung Chong & Haoen Cui & Yuxuan Li, 2021. "Pseudo-Model-Free Hedging for Variable Annuities via Deep Reinforcement Learning," Papers 2107.03340, arXiv.org, revised Oct 2022.
    17. Allan Jonathan da Silva & Jack Baczynski & Leonardo Fagundes de Mello, 2023. "Hedging Interest Rate Options with Reinforcement Learning: an investigation of a heavy-tailed distribution," Business and Management Studies, Redfame publishing, vol. 9(2), pages 1-14, December.
    18. Ariel Neufeld & Julian Sester & Mario Šikić, 2023. "Markov decision processes under model uncertainty," Mathematical Finance, Wiley Blackwell, vol. 33(3), pages 618-665, July.
    19. Zoran Stoiljkovic, 2023. "Applying Reinforcement Learning to Option Pricing and Hedging," Papers 2310.04336, arXiv.org.
    20. Christa Cuchiero & Guido Gazzani & Sara Svaluto-Ferro, 2022. "Signature-based models: theory and calibration," Papers 2207.13136, arXiv.org.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2401.08600. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.