IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1907.03665.html
   My bibliography  Save this paper

An intelligent financial portfolio trading strategy using deep Q-learning

Author

Listed:
  • Hyungjun Park
  • Min Kyu Sim
  • Dong Gu Choi

Abstract

Portfolio traders strive to identify dynamic portfolio allocation schemes so that their total budgets are well allocated through the investment horizon. This study proposes a novel portfolio trading strategy in which an intelligent agent is trained to identify an optimal trading action by using an algorithm called deep Q-learning. This study formulates a portfolio trading process as a Markov decision process in which the agent can learn about the financial market environment, and it identifies a deep neural network structure as an approximation of the Q-function. To ensure applicability to real-world trading, we devise three novel techniques that are both reasonable and implementable. First, the agent's action space is modeled as a combinatorial action space of trading directions with prespecified trading sizes for each asset. Second, we introduce a mapping function that can replace an initially-determined action that may be infeasible with a feasible action that is reasonably close to the original, ideal action. Last, we introduce a technique by which an agent simulates all feasible actions in each state and learns about these experiences to derive a multi-asset trading strategy that best reflects financial data. To validate our approach, we conduct backtests for two representative portfolios and demonstrate superior results over the benchmark strategies.

Suggested Citation

  • Hyungjun Park & Min Kyu Sim & Dong Gu Choi, 2019. "An intelligent financial portfolio trading strategy using deep Q-learning," Papers 1907.03665, arXiv.org, revised Aug 2019.
  • Handle: RePEc:arx:papers:1907.03665
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1907.03665
    File Function: Latest version
    Download Restriction: no

    References listed on IDEAS

    as
    1. Golub, Bennett & Holmer, Martin & McKendall, Raymond & Pohlman, Lawrence & Zenios, Stavros A., 1995. "A stochastic programming model for money management," European Journal of Operational Research, Elsevier, vol. 85(2), pages 282-296, September.
    2. Justin Sirignano & Rama Cont, 2018. "Universal features of price formation in financial markets: perspectives from Deep Learning," Papers 1803.06917, arXiv.org.
    3. Francesco Bertoluzzo & Marco Corazza, 2012. "Reinforcement Learning for automatic financial trading: Introduction and some applications," Working Papers 2012:33, Department of Economics, University of Venice "Ca' Foscari", revised 2012.
    4. Xufre Casqueiro, Patricia & Rodrigues, Antonio J.L., 2006. "Neuro-dynamic trading methods," European Journal of Operational Research, Elsevier, vol. 175(3), pages 1400-1412, December.
    5. Zhengyao Jiang & Dixing Xu & Jinjun Liang, 2017. "A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem," Papers 1706.10059, arXiv.org, revised Jul 2017.
    6. G. Consigli & M. Dempster, 1998. "Dynamic stochastic programmingfor asset-liability management," Annals of Operations Research, Springer, vol. 81(0), pages 131-162, June.
    7. Brock, William & Lakonishok, Josef & LeBaron, Blake, 1992. " Simple Technical Trading Rules and the Stochastic Properties of Stock Returns," Journal of Finance, American Finance Association, vol. 47(5), pages 1731-1764, December.
    8. Kouwenberg, Roy, 2001. "Scenario generation and stochastic programming models for asset liability management," European Journal of Operational Research, Elsevier, vol. 134(2), pages 279-292, October.
    9. Papailias, Fotis & Thomakos, Dimitrios D., 2015. "An improved moving average technical trading rule," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 428(C), pages 458-469.
    10. Justin Sirignano & Rama Cont, 2018. "Universal features of price formation in financial markets: perspectives from Deep Learning," Working Papers hal-01754054, HAL.
    11. Zhu, Yingzi & Zhou, Guofu, 2009. "Technical analysis: An asset allocation perspective on the use of moving averages," Journal of Financial Economics, Elsevier, vol. 92(3), pages 519-544, June.
    12. repec:eee:ejores:v:270:y:2018:i:2:p:654-669 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1907.03665. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (arXiv administrators). General contact details of provider: http://arxiv.org/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.