IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2302.07996.html
   My bibliography  Save this paper

A Comparison of Reinforcement Learning and Deep Trajectory Based Stochastic Control Agents for Stepwise Mean-Variance Hedging

Author

Listed:
  • Ali Fathi
  • Bernhard Hientzsch

Abstract

We consider two data-driven approaches to hedging, Reinforcement Learning and Deep Trajectory-based Stochastic Optimal Control, under a stepwise mean-variance objective. We compare their performance for a European call option in the presence of transaction costs under discrete trading schedules. We do this for a setting where stock prices follow Black-Scholes-Merton dynamics and the "book-keeping" price for the option is given by the Black-Scholes-Merton model with the same parameters. This simulated data setting provides a "sanitized" lab environment with simple enough features where we can conduct a detailed study of strengths, features, issues, and limitations of these two approaches. However, the formulation is model free and could allow any other setting with available book-keeping prices. We consider this study as a first step to develop, test, and validate autonomous hedging agents, and we provide blueprints for such efforts that address various concerns and requirements.

Suggested Citation

  • Ali Fathi & Bernhard Hientzsch, 2023. "A Comparison of Reinforcement Learning and Deep Trajectory Based Stochastic Control Agents for Stepwise Mean-Variance Hedging," Papers 2302.07996, arXiv.org, revised Nov 2023.
    • Handle: RePEc:arx:papers:2302.07996
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2302.07996
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Samuel N. Cohen & Derek Snow & Lukasz Szpruch, 2021. "Black-box model risk in finance," Papers 2102.04757, arXiv.org.
    2. Magnus Wiese & Lianjun Bai & Ben Wood & Hans Buehler, 2019. "Deep Hedging: Learning to Simulate Equity Option Markets," Papers 1911.01700, arXiv.org.
    3. Magnus Wiese & Phillip Murray, 2022. "Risk-Neutral Market Simulation," Papers 2202.13996, arXiv.org.
    4. Jay Cao & Jacky Chen & John Hull & Zissis Poulos, 2021. "Deep Hedging of Derivatives Using Reinforcement Learning," Papers 2103.16409, arXiv.org.
    5. Nicolas Boursin & Carl Remlinger & Joseph Mikael, 2022. "Deep Generators on Commodity Markets Application to Deep Hedging," Risks, MDPI, vol. 11(1), pages 1-18, December.
    6. Narayan Ganesan & Yajie Yu & Bernhard Hientzsch, 2020. "Pricing Barrier Options with DeepBSDEs," Papers 2005.10966, arXiv.org.
    7. A. Max Reppen & H. Mete Soner & Valentin Tissot-Daguette, 2022. "Deep Stochastic Optimization in Finance," Papers 2205.04604, arXiv.org.
    8. Jian Liang & Zhe Xu & Peter Li, 2021. "Deep learning-based least squares forward-backward stochastic differential equation solver for high-dimensional derivative pricing," Quantitative Finance, Taylor & Francis Journals, vol. 21(8), pages 1309-1323, August.
    9. Nicolas Boursin & Carl Remlinger & Joseph Mikael & Carol Anne Hargreaves, 2022. "Deep Generators on Commodity Markets; application to Deep Hedging," Papers 2205.13942, arXiv.org.
    10. Bernhard Hientzsch, 2019. "Introduction to Solving Quant Finance Problems with Time-Stepped FBSDE and Deep Learning," Papers 1911.12231, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bernhard Hientzsch, 2023. "Reinforcement Learning and Deep Stochastic Optimal Control for Final Quadratic Hedging," Papers 2401.08600, arXiv.org.
    2. Vedant Choudhary & Sebastian Jaimungal & Maxime Bergeron, 2023. "FuNVol: A Multi-Asset Implied Volatility Market Simulator using Functional Principal Components and Neural SDEs," Papers 2303.00859, arXiv.org, revised Dec 2023.
    3. Yajie Yu & Bernhard Hientzsch & Narayan Ganesan, 2020. "Backward Deep BSDE Methods and Applications to Nonlinear Problems," Papers 2006.07635, arXiv.org.
    4. Samuel N. Cohen & Christoph Reisinger & Sheng Wang, 2022. "Estimating risks of option books using neural-SDE market models," Papers 2202.07148, arXiv.org.
    5. Michael Karpe, 2020. "An overall view of key problems in algorithmic trading and recent progress," Papers 2006.05515, arXiv.org.
    6. Solveig Flaig & Gero Junike, 2022. "Scenario Generation for Market Risk Models Using Generative Neural Networks," Risks, MDPI, vol. 10(11), pages 1-28, October.
    7. Blanka Horvath & Josef Teichmann & Žan Žurič, 2021. "Deep Hedging under Rough Volatility," Risks, MDPI, vol. 9(7), pages 1-20, July.
    8. Alexandre Carbonneau & Fr'ed'eric Godin, 2021. "Deep equal risk pricing of financial derivatives with non-translation invariant risk measures," Papers 2107.11340, arXiv.org.
    9. Hans Buhler & Blanka Horvath & Terry Lyons & Imanol Perez Arribas & Ben Wood, 2020. "A Data-driven Market Simulator for Small Data Environments," Papers 2006.14498, arXiv.org.
    10. Federico Giorgi & Stefano Herzel & Paolo Pigato, 2023. "A Reinforcement Learning Algorithm for Trading Commodities," CEIS Research Paper 552, Tor Vergata University, CEIS, revised 18 Feb 2023.
    11. Jay Cao & Jacky Chen & Soroush Farghadani & John Hull & Zissis Poulos & Zeyu Wang & Jun Yuan, 2022. "Gamma and Vega Hedging Using Deep Distributional Reinforcement Learning," Papers 2205.05614, arXiv.org, revised Jan 2023.
    12. Ben Hambly & Renyuan Xu & Huining Yang, 2021. "Recent Advances in Reinforcement Learning in Finance," Papers 2112.04553, arXiv.org, revised Feb 2023.
    13. Blanka Horvath & Josef Teichmann & Zan Zuric, 2021. "Deep Hedging under Rough Volatility," Papers 2102.01962, arXiv.org.
    14. Lorenc Kapllani & Long Teng, 2024. "A backward differential deep learning-based algorithm for solving high-dimensional nonlinear backward stochastic differential equations," Papers 2404.08456, arXiv.org.
    15. Weilong Fu & Ali Hirsa & Jorg Osterrieder, 2022. "Simulating financial time series using attention," Papers 2207.00493, arXiv.org.
    16. Hans Buehler & Phillip Murray & Mikko S. Pakkanen & Ben Wood, 2021. "Deep Hedging: Learning Risk-Neutral Implied Volatility Dynamics," Papers 2103.11948, arXiv.org, revised Jul 2021.
    17. Pierre Renucci, 2023. "Optimal Linear Signal: An Unsupervised Machine Learning Framework to Optimize PnL with Linear Signals," Papers 2401.05337, arXiv.org.
    18. Josef Teichmann & Hanna Wutte, 2023. "Machine Learning-powered Pricing of the Multidimensional Passport Option," Papers 2307.14887, arXiv.org.
    19. Zacharia Issa & Blanka Horvath, 2023. "Non-parametric online market regime detection and regime clustering for multidimensional and path-dependent data structures," Papers 2306.15835, arXiv.org.
    20. Ariel Neufeld & Julian Sester & Mario v{S}iki'c, 2022. "Markov Decision Processes under Model Uncertainty," Papers 2206.06109, arXiv.org, revised Jan 2023.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2302.07996. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.