IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2004.06627.html
   My bibliography  Save this paper

An Application of Deep Reinforcement Learning to Algorithmic Trading

Author

Listed:
  • Thibaut Th'eate
  • Damien Ernst

Abstract

This scientific research paper presents an innovative approach based on deep reinforcement learning (DRL) to solve the algorithmic trading problem of determining the optimal trading position at any point in time during a trading activity in stock markets. It proposes a novel DRL trading strategy so as to maximise the resulting Sharpe ratio performance indicator on a broad range of stock markets. Denominated the Trading Deep Q-Network algorithm (TDQN), this new trading strategy is inspired from the popular DQN algorithm and significantly adapted to the specific algorithmic trading problem at hand. The training of the resulting reinforcement learning (RL) agent is entirely based on the generation of artificial trajectories from a limited set of stock market historical data. In order to objectively assess the performance of trading strategies, the research paper also proposes a novel, more rigorous performance assessment methodology. Following this new performance assessment approach, promising results are reported for the TDQN strategy.

Suggested Citation

  • Thibaut Th'eate & Damien Ernst, 2020. "An Application of Deep Reinforcement Learning to Algorithmic Trading," Papers 2004.06627, arXiv.org, revised Oct 2020.
    • Handle: RePEc:arx:papers:2004.06627
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2004.06627
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. John P A Ioannidis, 2005. "Why Most Published Research Findings Are False," PLOS Medicine, Public Library of Science, vol. 2(8), pages 1-1, August.
    2. Terrence Hendershott & Charles M. Jones & Albert J. Menkveld, 2011. "Does Algorithmic Trading Improve Liquidity?," Journal of Finance, American Finance Association, vol. 66(1), pages 1-33, February.
    3. Wei Bao & Jun Yue & Yulei Rao, 2017. "A deep learning framework for financial time series using stacked autoencoders and long-short term memory," PLOS ONE, Public Library of Science, vol. 12(7), pages 1-24, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Eric Benhamou & David Saltiel & Sandrine Ungari & Abhishek Mukhopadhyay & Jamal Atif, 2020. "AAMDRL: Augmented Asset Management with Deep Reinforcement Learning," Papers 2010.08497, arXiv.org.
    2. Thibaut Théate & Sébastien Mathieu & Damien Ernst, 2020. "An Artificial Intelligence Solution for Electricity Procurement in Forward Markets," Energies, MDPI, vol. 13(23), pages 1-17, December.
    3. Eric Benhamou & David Saltiel & Sandrine Ungari & Abhishek Mukhopadhyay, 2020. "Bridging the gap between Markowitz planning and deep reinforcement learning," Papers 2010.09108, arXiv.org.
    4. Antonio Briola & Jeremy Turiel & Riccardo Marcaccioli & Alvaro Cauderan & Tomaso Aste, 2021. "Deep Reinforcement Learning for Active High Frequency Trading," Papers 2101.07107, arXiv.org, revised Aug 2023.
    5. Peng Chen & Dongyun Yi & Chengli Zhao, 2020. "Trading Strategy for Market Situation Estimation Based on Hidden Markov Model," Mathematics, MDPI, vol. 8(7), pages 1-13, July.
    6. Tatiana de Macedo Nogueira Lima, 2022. "Documento de Trabalho 03/2022 - Aprendizado de máquina e antitruste," Documentos de Trabalho 2022030, Conselho Administrativo de Defesa Econômica (Cade), Departamento de Estudos Econômicos.
    7. Thibaut Th'eate & S'ebastien Mathieu & Damien Ernst, 2020. "An Artificial Intelligence Solution for Electricity Procurement in Forward Markets," Papers 2006.05784, arXiv.org, revised Dec 2020.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zihao Zhang & Stefan Zohren & Stephen Roberts, 2018. "DeepLOB: Deep Convolutional Neural Networks for Limit Order Books," Papers 1808.03668, arXiv.org, revised Jan 2020.
    2. Benos, Evangelos & Sagade, Satchit, 2012. "High-frequency trading behaviour and its impact on market quality: evidence from the UK equity market," Bank of England working papers 469, Bank of England.
    3. Bellia, Mario & Christensen, Kim & Kolokolov, Aleksey & Pelizzon, Loriana & Renò, Roberto, 2022. "Do designated market makers provide liquidity during a flash crash?," SAFE Working Paper Series 270, Leibniz Institute for Financial Research SAFE, revised 2022.
    4. Tamer Khraisha & Keren Arthur, 2018. "Can we have a general theory of financial innovation processes? A conceptual review," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 4(1), pages 1-27, December.
    5. Alexander Frankel & Maximilian Kasy, 2022. "Which Findings Should Be Published?," American Economic Journal: Microeconomics, American Economic Association, vol. 14(1), pages 1-38, February.
    6. Bruno Biais & Fany Declerck & Sophie Moinas, 2016. "Who supplies liquidity, how and when?," BIS Working Papers 563, Bank for International Settlements.
    7. Jyotirmoy Sarkar, 2018. "Will P†Value Triumph over Abuses and Attacks?," Biostatistics and Biometrics Open Access Journal, Juniper Publishers Inc., vol. 7(4), pages 66-71, July.
    8. Indriawan, Ivan & Martinez, Valeria & Tse, Yiuman, 2021. "The impact of the change in USDA announcement release procedures on agricultural commodity futures," Journal of Commodity Markets, Elsevier, vol. 23(C).
    9. Andrea Bucci, 2020. "Realized Volatility Forecasting with Neural Networks," Journal of Financial Econometrics, Oxford University Press, vol. 18(3), pages 502-531.
    10. Álvaro Cartea & José Penalva, 2012. "Where is the Value in High Frequency Trading?," Quarterly Journal of Finance (QJF), World Scientific Publishing Co. Pte. Ltd., vol. 2(03), pages 1-46.
    11. NIdhi Aggarwal & Venkatesh Panchapagesan & Susan Thomas, 2022. "When is the Order to Trade Ratio fee effective?," Working Papers 8, xKDR.
    12. Kang, Jongho & Kang, Jangkoo & Kwon, Kyung Yoon, 2022. "Market versus limit orders of speculative high-frequency traders and price discovery," Research in International Business and Finance, Elsevier, vol. 63(C).
    13. Ahmed Baig & Nasim Sabah & Drew Winters, 2019. "Have Stock Prices become more Uniformly Distributed?," Economics Bulletin, AccessEcon, vol. 39(2), pages 1242-1250.
    14. Robert J. Kauffman & Yuzhou Hu & Dan Ma, 2015. "Will high-frequency trading practices transform the financial markets in the Asia Pacific Region?," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 1(1), pages 1-27, December.
    15. Uctum, Remzi & Renou-Maissant, Patricia & Prat, Georges & Lecarpentier-Moyal, Sylvie, 2017. "Persistence of announcement effects on the intraday volatility of stock returns: Evidence from individual data," Review of Financial Economics, Elsevier, vol. 35(C), pages 43-56.
    16. Stanley, T. D. & Doucouliagos, Chris, 2019. "Practical Significance, Meta-Analysis and the Credibility of Economics," IZA Discussion Papers 12458, Institute of Labor Economics (IZA).
    17. George Jiang & Ingrid Lo & Giorgio Valente, 2014. "High-Frequency Trading around Macroeconomic News Announcements: Evidence from the U.S. Treasury Market," Staff Working Papers 14-56, Bank of Canada.
    18. Jaydip Sen & Sidra Mehtab & Abhishek Dutta & Saikat Mondal, 2022. "Precise Stock Price Prediction for Optimized Portfolio Design Using an LSTM Model," Papers 2203.01326, arXiv.org.
    19. Aggarwal, Nidhi & Panchapagesan, Venkatesh & Thomas, Susan, 2023. "When is the order-to-trade ratio fee effective?," Journal of Financial Markets, Elsevier, vol. 62(C).
    20. Bank, Matthias & Baumann, Ralf H., 2016. "Price formation, market quality and the effects of reduced latency in the very short run," Research in International Business and Finance, Elsevier, vol. 37(C), pages 629-645.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2004.06627. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.