IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2003.06497.html
   My bibliography  Save this paper

Deep Deterministic Portfolio Optimization

Author

Listed:
  • Ayman Chaouki
  • Stephen Hardiman
  • Christian Schmidt
  • Emmanuel S'eri'e
  • Joachim de Lataillade

Abstract

Can deep reinforcement learning algorithms be exploited as solvers for optimal trading strategies? The aim of this work is to test reinforcement learning algorithms on conceptually simple, but mathematically non-trivial, trading environments. The environments are chosen such that an optimal or close-to-optimal trading strategy is known. We study the deep deterministic policy gradient algorithm and show that such a reinforcement learning agent can successfully recover the essential features of the optimal trading strategies and achieve close-to-optimal rewards.

Suggested Citation

  • Ayman Chaouki & Stephen Hardiman & Christian Schmidt & Emmanuel S'eri'e & Joachim de Lataillade, 2020. "Deep Deterministic Portfolio Optimization," Papers 2003.06497, arXiv.org, revised Apr 2020.
    • Handle: RePEc:arx:papers:2003.06497
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2003.06497
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Chamberlain, Gary, 1983. "A characterization of the distributions that imply mean--Variance utility functions," Journal of Economic Theory, Elsevier, vol. 29(1), pages 185-201, February.
    2. Richard Martin & Torsten Schoneborn, 2011. "Mean Reversion Pays, but Costs," Papers 1103.4934, arXiv.org.
    3. Xiao-Yang Liu & Zhuoran Xiong & Shan Zhong & Hongyang Yang & Anwar Walid, 2018. "Practical Deep Reinforcement Learning Approach for Stock Trading," Papers 1811.07522, arXiv.org, revised Jul 2022.
    4. Nicolae Gârleanu & Lasse Heje Pedersen, 2013. "Dynamic Trading with Predictable Returns and Transaction Costs," Journal of Finance, American Finance Association, vol. 68(6), pages 2309-2340, December.
    5. Johannes Muhle-Karbe & Max Reppen & H. Mete Soner, 2016. "A Primer on Portfolio Choice with Small Transaction Costs," Papers 1612.01302, arXiv.org, revised May 2017.
    6. Merton, Robert C, 1969. "Lifetime Portfolio Selection under Uncertainty: The Continuous-Time Case," The Review of Economics and Statistics, MIT Press, vol. 51(3), pages 247-257, August.
    7. Bertsimas, Dimitris & Lo, Andrew W., 1998. "Optimal control of execution costs," Journal of Financial Markets, Elsevier, vol. 1(1), pages 1-50, April.
    8. Joachim de Lataillade & Cyril Deremble & Marc Potters & Jean-Philippe Bouchaud, 2012. "Optimal Trading with Linear Costs," Papers 1203.5957, arXiv.org.
    9. David Silver & Julian Schrittwieser & Karen Simonyan & Ioannis Antonoglou & Aja Huang & Arthur Guez & Thomas Hubert & Lucas Baker & Matthew Lai & Adrian Bolton & Yutian Chen & Timothy Lillicrap & Fan , 2017. "Mastering the game of Go without human knowledge," Nature, Nature, vol. 550(7676), pages 354-359, October.
    10. M. Abeille & E. Serie & A. Lazaric & X. Brokmann, 2016. "LQG for portfolio optimization," Papers 1611.00997, arXiv.org, revised Nov 2016.
    11. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    12. Stephen Boyd & Enzo Busseti & Steven Diamond & Ronald N. Kahn & Kwangmoo Koh & Peter Nystrup & Jan Speth, 2017. "Multi-Period Trading via Convex Optimization," Papers 1705.00109, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Joachim de Lataillade & Ayman Chaouki, 2020. "Equations and Shape of the Optimal Band Strategy," Papers 2003.04646, arXiv.org, revised Mar 2020.
    2. Alessio Brini & Daniele Tantari, 2021. "Deep Reinforcement Trading with Predictable Returns," Papers 2104.14683, arXiv.org, revised May 2023.
    3. Brini, Alessio & Tantari, Daniele, 2023. "Deep reinforcement trading with predictable returns," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 622(C).
    4. Karush Suri & Xiao Qi Shi & Konstantinos Plataniotis & Yuri Lawryshyn, 2021. "TradeR: Practical Deep Hierarchical Reinforcement Learning for Trade Execution," Papers 2104.00620, arXiv.org.
    5. Thibault Jaisson, 2021. "Deep differentiable reinforcement learning and optimal trading," Papers 2112.02944, arXiv.org, revised Apr 2022.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ibrahim Ekren & Johannes Muhle-Karbe, 2017. "Portfolio Choice with Small Temporary and Transient Price Impact," Papers 1705.00672, arXiv.org, revised Apr 2020.
    2. Florent Gallien & Serge Kassibrakis & Semyon Malamud, 2018. "Hedge or Rebalance: Optimal Risk Management with Transaction Costs," Risks, MDPI, vol. 6(4), pages 1-14, October.
    3. Joachim de Lataillade & Ayman Chaouki, 2020. "Equations and Shape of the Optimal Band Strategy," Papers 2003.04646, arXiv.org, revised Mar 2020.
    4. Jan Kallsen & Johannes Muhle-Karbe, 2013. "The General Structure of Optimal Investment and Consumption with Small Transaction Costs," Papers 1303.3148, arXiv.org, revised May 2015.
    5. Xiaoyue Li & A. Sinem Uysal & John M. Mulvey, 2021. "Multi-Period Portfolio Optimization using Model Predictive Control with Mean-Variance and Risk Parity Frameworks," Papers 2103.10813, arXiv.org.
    6. Johannes Muhle-Karbe & Xiaofei Shi & Chen Yang, 2020. "An Equilibrium Model for the Cross-Section of Liquidity Premia," Papers 2011.13625, arXiv.org.
    7. Johannes Muhle-Karbe & Max Reppen & H. Mete Soner, 2016. "A Primer on Portfolio Choice with Small Transaction Costs," Papers 1612.01302, arXiv.org, revised May 2017.
    8. Filippo Passerini & Samuel E. Vazquez, 2015. "Optimal Trading with Alpha Predictors," Papers 1501.03756, arXiv.org, revised Jan 2015.
    9. Ren Liu & Johannes Muhle-Karbe & Marko H. Weber, 2014. "Rebalancing with Linear and Quadratic Costs," Papers 1402.5306, arXiv.org, revised Sep 2017.
    10. Puru Gupta & Saul D. Jacka, 2023. "Portfolio Choice In Dynamic Thin Markets: Merton Meets Cournot," Papers 2309.16047, arXiv.org.
    11. Shuo Sun & Rundong Wang & Bo An, 2021. "Reinforcement Learning for Quantitative Trading," Papers 2109.13851, arXiv.org.
    12. Lukas Gonon & Johannes Muhle-Karbe & Xiaofei Shi, 2019. "Asset Pricing with General Transaction Costs: Theory and Numerics," Papers 1905.05027, arXiv.org, revised Apr 2020.
    13. Martin Herdegen & Johannes Muhle-Karbe & Dylan Possamai, 2019. "Equilibrium Asset Pricing with Transaction Costs," Papers 1901.10989, arXiv.org, revised Sep 2020.
    14. Martin Herdegen & Johannes Muhle-Karbe & Dylan Possamaï, 2021. "Equilibrium asset pricing with transaction costs," Finance and Stochastics, Springer, vol. 25(2), pages 231-275, April.
    15. Matt Emschwiller & Benjamin Petit & Jean-Philippe Bouchaud, 2019. "Optimal multi-asset trading with linear costs: a mean-field approach," Papers 1905.04821, arXiv.org, revised Apr 2020.
    16. Ludovic Moreau & Johannes Muhle-Karbe & H. Mete Soner, 2014. "Trading with Small Price Impact," Papers 1402.5304, arXiv.org, revised Mar 2015.
    17. M. Abeille & E. Serie & A. Lazaric & X. Brokmann, 2016. "LQG for portfolio optimization," Papers 1611.00997, arXiv.org, revised Nov 2016.
    18. Zechu Li & Xiao-Yang Liu & Jiahao Zheng & Zhaoran Wang & Anwar Walid & Jian Guo, 2021. "FinRL-Podracer: High Performance and Scalable Deep Reinforcement Learning for Quantitative Finance," Papers 2111.05188, arXiv.org.
    19. Richard J. Martin, 2012. "Optimal multifactor trading under proportional transaction costs," Papers 1204.6488, arXiv.org.
    20. Yan, Tingjin & Han, Jinhui & Ma, Guiyuan & Siu, Chi Chung, 2023. "Dynamic asset-liability management with frictions," Insurance: Mathematics and Economics, Elsevier, vol. 111(C), pages 57-83.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2003.06497. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.