IDEAS home Printed from https://ideas.repec.org/a/spr/finsto/v26y2022i1d10.1007_s00780-021-00467-2.html
   My bibliography  Save this article

Reinforcement learning and stochastic optimisation

Author

Listed:
  • Sebastian Jaimungal

    (University of Toronto)

Abstract

At the heart of financial mathematics lie stochastic optimisation problems. Traditional approaches to solving such problems, while applicable to broad classes of models, require specifying a model to complete the analysis and obtain implementable results. Even then, the curse of dimensionality challenges the viability of conventional methods to settings of practical relevance. In contrast, machine learning, and reinforcement learning (RL) particularly, promises to learn from data and overcome the curse of dimensionality simultaneously. This article touches on several approaches in the extant literature that are well positioned to merge our traditional techniques with RL.

Suggested Citation

  • Sebastian Jaimungal, 2022. "Reinforcement learning and stochastic optimisation," Finance and Stochastics, Springer, vol. 26(1), pages 103-129, January.
    • Handle: RePEc:spr:finsto:v:26:y:2022:i:1:d:10.1007_s00780-021-00467-2
    DOI: 10.1007/s00780-021-00467-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00780-021-00467-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00780-021-00467-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Anastasia Borovykh & Andrea Pascucci & Stefano La Rovere, 2018. "Systemic risk in a mean-field model of interbank lending with self-exciting shocks," IISE Transactions, Taylor & Francis Journals, vol. 50(9), pages 806-819, September.
    2. Christian Bayer & Blanka Horvath & Aitor Muguruza & Benjamin Stemper & Mehdi Tomas, 2019. "On deep calibration of (rough) stochastic volatility models," Papers 1908.08806, arXiv.org.
    3. Kreps, David M., 1981. "Arbitrage and equilibrium in economies with infinitely many commodities," Journal of Mathematical Economics, Elsevier, vol. 8(1), pages 15-35, March.
    4. Steven Campbell & Yichao Chen & Arvind Shrivats & Sebastian Jaimungal, 2021. "Deep Learning for Principal-Agent Mean Field Games," Papers 2110.01127, arXiv.org.
    5. Ali Al-Aradi & Sebastian Jaimungal, 2018. "Outperformance and Tracking: Dynamic Asset Allocation for Active and Passive Portfolio Management," Applied Mathematical Finance, Taylor & Francis Journals, vol. 25(3), pages 268-294, May.
    6. Ali Al-Aradi & Adolfo Correia & Danilo Naiff & Gabriel Jardim & Yuri Saporito, 2018. "Solving Nonlinear and High-Dimensional Partial Differential Equations via Deep Learning," Papers 1811.08782, arXiv.org.
    7. Robert C. Merton, 2005. "Theory of rational option pricing," World Scientific Book Chapters, in: Sudipto Bhattacharya & George M Constantinides (ed.), Theory Of Valuation, chapter 8, pages 229-288, World Scientific Publishing Co. Pte. Ltd..
    8. Christa Cuchiero & Wahid Khosrawi & Josef Teichmann, 2020. "A Generative Adversarial Network Approach to Calibration of Local Stochastic Volatility Models," Risks, MDPI, vol. 8(4), pages 1-31, September.
    9. Justin Sirignano & Rama Cont, 2019. "Universal features of price formation in financial markets: perspectives from deep learning," Quantitative Finance, Taylor & Francis Journals, vol. 19(9), pages 1449-1459, September.
    10. Jiequn Han & Ruimeng Hu, 2019. "Deep Fictitious Play for Finding Markovian Nash Equilibrium in Multi-Agent Games," Papers 1912.01809, arXiv.org, revised Jun 2020.
    11. Christa Cuchiero & Wahid Khosrawi & Josef Teichmann, 2020. "A generative adversarial network approach to calibration of local stochastic volatility models," Papers 2005.02505, arXiv.org, revised Sep 2020.
    12. Maximilien Germain & Huyên Pham & Xavier Warin, 2021. "Neural networks-based algorithms for stochastic control and PDEs in finance ," Working Papers hal-03115503, HAL.
    13. Ali Al-Aradi & Sebastian Jaimungal, 2018. "Outperformance and Tracking: Dynamic Asset Allocation for Active and Passive Portfolio Management," Papers 1803.05819, arXiv.org, revised Jul 2018.
    14. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    15. Sirignano, Justin & Spiliopoulos, Konstantinos, 2020. "Mean field analysis of neural networks: A central limit theorem," Stochastic Processes and their Applications, Elsevier, vol. 130(3), pages 1820-1852.
    16. Charles-Albert Lehalle & Eyal Neuman, 2019. "Incorporating signals into optimal trading," Finance and Stochastics, Springer, vol. 23(2), pages 275-311, April.
    17. Chendi Ni & Yuying Li & Peter Forsyth & Ray Carroll, 2020. "Optimal Asset Allocation For Outperforming A Stochastic Benchmark Target," Papers 2006.15384, arXiv.org.
    18. Xuancheng Huang & Sebastian Jaimungal & Mojtaba Nourian, 2019. "Mean-Field Game Strategies for Optimal Execution," Applied Mathematical Finance, Taylor & Francis Journals, vol. 26(2), pages 153-185, March.
    19. Dena Firoozi & Arvind V Shrivats & Sebastian Jaimungal, 2021. "Principal agent mean field games in REC markets," Papers 2112.11963, arXiv.org, revised Jun 2022.
    20. Olivier Féron & Peter Tankov & Laura Tinsi, 2020. "Price Formation and Optimal Trading in Intraday Electricity Markets with a Major Player," Risks, MDPI, vol. 8(4), pages 1-21, December.
    21. Patryk Gierjatowicz & Marc Sabate-Vidales & David v{S}iv{s}ka & Lukasz Szpruch & v{Z}an v{Z}uriv{c}, 2020. "Robust pricing and hedging via neural SDEs," Papers 2007.04154, arXiv.org.
    22. Arvind Shrivats & Dena Firoozi & Sebastian Jaimungal, 2020. "A Mean-Field Game Approach to Equilibrium Pricing in Solar Renewable Energy Certificate Markets," Papers 2003.04938, arXiv.org, revised Aug 2021.
    23. Harrison, J. Michael & Kreps, David M., 1979. "Martingales and arbitrage in multiperiod securities markets," Journal of Economic Theory, Elsevier, vol. 20(3), pages 381-408, June.
    24. Harrison, J. Michael & Pliska, Stanley R., 1981. "Martingales and stochastic integrals in the theory of continuous trading," Stochastic Processes and their Applications, Elsevier, vol. 11(3), pages 215-260, August.
    25. Olivier F'eron & Peter Tankov & Laura Tinsi, 2020. "Price formation and optimal trading in intraday electricity markets with a major player," Papers 2011.07655, arXiv.org.
    26. Philippe Casgrain & Brian Ning & Sebastian Jaimungal, 2019. "Deep Q-Learning for Nash Equilibria: Nash-DQN," Papers 1904.10554, arXiv.org, revised Oct 2022.
    27. Jan De Spiegeleer & Dilip B. Madan & Sofie Reyners & Wim Schoutens, 2018. "Machine learning for quantitative finance: fast derivative pricing, hedging and fitting," Quantitative Finance, Taylor & Francis Journals, vol. 18(10), pages 1635-1643, October.
    28. Philippe Casgrain & Sebastian Jaimungal, 2019. "Trading algorithms with learning in latent alpha models," Mathematical Finance, Wiley Blackwell, vol. 29(3), pages 735-772, July.
    29. Black, Fischer & Scholes, Myron S, 1973. "The Pricing of Options and Corporate Liabilities," Journal of Political Economy, University of Chicago Press, vol. 81(3), pages 637-654, May-June.
    30. Charles-Albert Lehalle & Charafeddine Mouzouni, 2019. "A mean field game of portfolio trading and its consequences on perceived correlations," Working Papers hal-02003143, HAL.
    31. Ryan Ferguson & Andrew Green, 2018. "Deeply Learning Derivatives," Papers 1809.02233, arXiv.org, revised Oct 2018.
    32. Philippe Casgrain & Sebastian Jaimungal, 2020. "Mean‐field games with differing beliefs for algorithmic trading," Mathematical Finance, Wiley Blackwell, vol. 30(3), pages 995-1034, July.
    33. Ormerod, J. T. & Wand, M. P., 2010. "Explaining Variational Approximations," The American Statistician, American Statistical Association, vol. 64(2), pages 140-153.
    34. Sebastian Jaimungal & Silvana Pesenti & Ye Sheng Wang & Hariom Tatsat, 2021. "Robust Risk-Aware Reinforcement Learning," Papers 2108.10403, arXiv.org, revised Dec 2021.
    35. Maximilien Germain & Huy^en Pham & Xavier Warin, 2021. "Neural networks-based algorithms for stochastic control and PDEs in finance," Papers 2101.08068, arXiv.org, revised Apr 2021.
    36. Justin Sirignano & Konstantinos Spiliopoulos, 2017. "DGM: A deep learning algorithm for solving partial differential equations," Papers 1708.07469, arXiv.org, revised Sep 2018.
    37. Tadeu A. Ferreira, 2020. "Reinforced Deep Markov Models With Applications in Automatic Trading," Papers 2011.04391, arXiv.org.
    38. Maximilien Germain & Huyên Pham & Xavier Warin, 2021. "Neural networks-based algorithms for stochastic control and PDEs in finance ," Post-Print hal-03115503, HAL.
    39. Yuri F. Saporito & Zhaoyu Zhang, 2020. "PDGM: a Neural Network Approach to Solve Path-Dependent Partial Differential Equations," Papers 2003.02035, arXiv.org, revised Apr 2020.
    40. A. Al-Aradi & S. Jaimungal, 2021. "Active and passive portfolio management with latent factors," Quantitative Finance, Taylor & Francis Journals, vol. 21(9), pages 1437-1459, September.
    41. Hans Buhler & Blanka Horvath & Terry Lyons & Imanol Perez Arribas & Ben Wood, 2020. "A Data-driven Market Simulator for Small Data Environments," Papers 2006.14498, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Anthony Coache & Sebastian Jaimungal, 2021. "Reinforcement Learning with Dynamic Convex Risk Measures," Papers 2112.13414, arXiv.org, revised Nov 2022.
    2. Nelson Vadori, 2022. "Calibration of Derivative Pricing Models: a Multi-Agent Reinforcement Learning Perspective," Papers 2203.06865, arXiv.org, revised Oct 2023.
    3. Anthony Coache & Sebastian Jaimungal & 'Alvaro Cartea, 2022. "Conditionally Elicitable Dynamic Risk Measures for Deep Reinforcement Learning," Papers 2206.14666, arXiv.org, revised May 2023.
    4. Anastasis Kratsios & Cody Hyndman, 2023. "Generative Ornstein-Uhlenbeck Markets via Geometric Deep Learning," Papers 2302.09176, arXiv.org.
    5. Kristoffer Andersson & Cornelis W. Oosterlee, 2023. "D-TIPO: Deep time-inconsistent portfolio optimization with stocks and options," Papers 2308.10556, arXiv.org, revised Sep 2023.
    6. Ryan Donnelly & Sebastian Jaimungal, 2022. "Exploratory Control with Tsallis Entropy for Latent Factor Models," Papers 2211.07622, arXiv.org, revised Jan 2024.
    7. Phillip Murray & Ben Wood & Hans Buehler & Magnus Wiese & Mikko S. Pakkanen, 2022. "Deep Hedging: Continuous Reinforcement Learning for Hedging of General Portfolios across Multiple Risk Aversions," Papers 2207.07467, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Beatrice Acciaio & Anastasis Kratsios & Gudmund Pammer, 2022. "Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformer," Papers 2201.13094, arXiv.org, revised Mar 2023.
    2. Lukas Gonon, 2022. "Deep neural network expressivity for optimal stopping problems," Papers 2210.10443, arXiv.org.
    3. Samuel N. Cohen & Christoph Reisinger & Sheng Wang, 2021. "Arbitrage-free neural-SDE market models," Papers 2105.11053, arXiv.org, revised Aug 2021.
    4. Lukas Gonon, 2021. "Random feature neural networks learn Black-Scholes type PDEs without curse of dimensionality," Papers 2106.08900, arXiv.org.
    5. Carl Remlinger & Joseph Mikael & Romuald Elie, 2022. "Robust Operator Learning to Solve PDE," Working Papers hal-03599726, HAL.
    6. Masaaki Fujii & Akihiko Takahashi, 2021. "Equilibrium Price Formation with a Major Player and its Mean Field Limit," CARF F-Series CARF-F-509, Center for Advanced Research in Finance, Faculty of Economics, The University of Tokyo.
    7. Bjork, Tomas, 2009. "Arbitrage Theory in Continuous Time," OUP Catalogue, Oxford University Press, edition 3, number 9780199574742.
    8. W. Schachermayer, 1994. "Martingale Measures For Discrete‐Time Processes With Infinite Horizon," Mathematical Finance, Wiley Blackwell, vol. 4(1), pages 25-55, January.
    9. Masaaki Fujii & Akihiko Takahashi, 2021. "Equilibrium Price Formation with a Major Player and its Mean Field Limit," Papers 2102.10756, arXiv.org, revised Feb 2022.
    10. Duffie, Darrell, 2003. "Intertemporal asset pricing theory," Handbook of the Economics of Finance, in: G.M. Constantinides & M. Harris & R. M. Stulz (ed.), Handbook of the Economics of Finance, edition 1, volume 1, chapter 11, pages 639-742, Elsevier.
    11. Fred Espen Benth & Nils Detering & Silvia Lavagnini, 2021. "Accuracy of deep learning in calibrating HJM forward curves," Digital Finance, Springer, vol. 3(3), pages 209-248, December.
    12. Fred Espen Benth & Nils Detering & Silvia Lavagnini, 2020. "Accuracy of Deep Learning in Calibrating HJM Forward Curves," Papers 2006.01911, arXiv.org, revised May 2021.
    13. Keith A. Lewis, 2019. "A Simple Proof of the Fundamental Theorem of Asset Pricing," Papers 1912.01091, arXiv.org.
    14. Antoine Jacquier & Zan Zuric, 2023. "Random neural networks for rough volatility," Papers 2305.01035, arXiv.org.
    15. Laurens Van Mieghem & Antonis Papapantoleon & Jonas Papazoglou-Hennig, 2023. "Machine learning for option pricing: an empirical investigation of network architectures," Papers 2307.07657, arXiv.org.
    16. Masaaki Fujii & Akihiko Takahashi, 2021. "``Equilibrium Price Formation with a Major Player and its Mean Field Limit''," CIRJE F-Series CIRJE-F-1162, CIRJE, Faculty of Economics, University of Tokyo.
    17. Matteo Burzoni & Marco Frittelli & Zhaoxu Hou & Marco Maggis & Jan Obłój, 2019. "Pointwise Arbitrage Pricing Theory in Discrete Time," Mathematics of Operations Research, INFORMS, vol. 44(3), pages 1034-1057, August.
    18. Mark Broadie & Jerome B. Detemple, 2004. "ANNIVERSARY ARTICLE: Option Pricing: Valuation Models and Applications," Management Science, INFORMS, vol. 50(9), pages 1145-1177, September.
    19. Christa Cuchiero & Guido Gazzani & Janka Moller & Sara Svaluto-Ferro, 2023. "Joint calibration to SPX and VIX options with signature-based models," Papers 2301.13235, arXiv.org.
    20. Christa Cuchiero & Josef Teichmann, 2015. "A convergence result for the Emery topology and a variant of the proof of the fundamental theorem of asset pricing," Finance and Stochastics, Springer, vol. 19(4), pages 743-761, October.

    More about this item

    Keywords

    Stochastic optimisation; Stochastic games; Reinforcement learning; Machine learning;
    All these keywords.

    JEL classification:

    • C02 - Mathematical and Quantitative Methods - - General - - - Mathematical Economics
    • G11 - Financial Economics - - General Financial Markets - - - Portfolio Choice; Investment Decisions
    • G12 - Financial Economics - - General Financial Markets - - - Asset Pricing; Trading Volume; Bond Interest Rates

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:finsto:v:26:y:2022:i:1:d:10.1007_s00780-021-00467-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.