IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1907.01503.html
   My bibliography  Save this paper

Optimistic Bull or Pessimistic Bear: Adaptive Deep Reinforcement Learning for Stock Portfolio Allocation

Author

Listed:
  • Xinyi Li
  • Yinchuan Li
  • Yuancheng Zhan
  • Xiao-Yang Liu

Abstract

Portfolio allocation is crucial for investment companies. However, getting the best strategy in a complex and dynamic stock market is challenging. In this paper, we propose a novel Adaptive Deep Deterministic Reinforcement Learning scheme (Adaptive DDPG) for the portfolio allocation task, which incorporates optimistic or pessimistic deep reinforcement learning that is reflected in the influence from prediction errors. Dow Jones 30 component stocks are selected as our trading stocks and their daily prices are used as the training and testing data. We train the Adaptive DDPG agent and obtain a trading strategy. The Adaptive DDPG's performance is compared with the vanilla DDPG, Dow Jones Industrial Average index and the traditional min-variance and mean-variance portfolio allocation strategies. Adaptive DDPG outperforms the baselines in terms of the investment return and the Sharpe ratio.

Suggested Citation

  • Xinyi Li & Yinchuan Li & Yuancheng Zhan & Xiao-Yang Liu, 2019. "Optimistic Bull or Pessimistic Bear: Adaptive Deep Reinforcement Learning for Stock Portfolio Allocation," Papers 1907.01503, arXiv.org.
    • Handle: RePEc:arx:papers:1907.01503
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1907.01503
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Xiao-Yang Liu & Zhuoran Xiong & Shan Zhong & Hongyang Yang & Anwar Walid, 2018. "Practical Deep Reinforcement Learning Approach for Stock Trading," Papers 1811.07522, arXiv.org, revised Jul 2022.
    2. Wenhang Bao & Xiao-yang Liu, 2019. "Multi-Agent Deep Reinforcement Learning for Liquidation Strategy Analysis," Papers 1906.11046, arXiv.org.
    3. Germain Lefebvre & Maël Lebreton & Florent Meyniel & Sacha Bourgeois-Gironde & Stefano Palminteri, 2017. "Behavioural and neural characterization of optimistic reinforcement learning," Nature Human Behaviour, Nature, vol. 1(4), pages 1-9, April.
    4. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    5. Mark J. Flannery & Aris A. Protopapadakis, 2002. "Macroeconomic Factors Do Influence Aggregate Stock Returns," The Review of Financial Studies, Society for Financial Studies, vol. 15(3), pages 751-782.
    6. Myles E. Mangram, 2013. "A Simplified Perspective Of The Markowitz Portfolio Theory," Global Journal of Business Research, The Institute for Business and Finance Research, vol. 7(1), pages 59-70.
    7. Xinyi Li & Yinchuan Li & Xiao-Yang Liu & Christina Dan Wang, 2019. "Risk Management via Anomaly Circumvent: Mnemonic Deep Learning for Midterm Stock Prediction," Papers 1908.01112, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Eric Benhamou & David Saltiel & Serge Tabachnik & Sui Kai Wong & François Chareyron, 2021. "Distinguish the indistinguishable: a Deep Reinforcement Learning approach for volatility targeting models," Working Papers hal-03202431, HAL.
    2. Eric Benhamou & David Saltiel & Serge Tabachnik & Sui Kai Wong & Franc{c}ois Chareyron, 2021. "Adaptive learning for financial markets mixing model-based and model-free RL for volatility targeting," Papers 2104.10483, arXiv.org, revised Apr 2021.
    3. Eric Benhamou & David Saltiel & Sandrine Ungari & Abhishek Mukhopadhyay & Jamal Atif, 2020. "AAMDRL: Augmented Asset Management with Deep Reinforcement Learning," Papers 2010.08497, arXiv.org.
    4. Zechu Li & Xiao-Yang Liu & Jiahao Zheng & Zhaoran Wang & Anwar Walid & Jian Guo, 2021. "FinRL-Podracer: High Performance and Scalable Deep Reinforcement Learning for Quantitative Finance," Papers 2111.05188, arXiv.org.
    5. Eric Benhamou & David Saltiel & Sandrine Ungari & Abhishek Mukhopadhyay, 2020. "Bridging the gap between Markowitz planning and deep reinforcement learning," Papers 2010.09108, arXiv.org.
    6. Jingyi Gu & Sarvesh Shukla & Junyi Ye & Ajim Uddin & Guiling Wang, 2023. "Deep learning model with sentiment score and weekend effect in stock price prediction," SN Business & Economics, Springer, vol. 3(7), pages 1-20, July.
    7. Huifang Huang & Ting Gao & Yi Gui & Jin Guo & Peng Zhang, 2022. "Stock Trading Optimization through Model-based Reinforcement Learning with Resistance Support Relative Strength," Papers 2205.15056, arXiv.org.
    8. Xinyi Li & Yinchuan Li & Xiao-Yang Liu & Christina Dan Wang, 2019. "Risk Management via Anomaly Circumvent: Mnemonic Deep Learning for Midterm Stock Prediction," Papers 1908.01112, arXiv.org.
    9. Eric Benhamou & David Saltiel & Sandrine Ungari & Abhishek Mukhopadhyay, 2020. "Time your hedge with Deep Reinforcement Learning," Papers 2009.14136, arXiv.org, revised Nov 2020.
    10. Mei-Li Shen & Cheng-Feng Lee & Hsiou-Hsiang Liu & Po-Yin Chang & Cheng-Hong Yang, 2021. "An Effective Hybrid Approach for Forecasting Currency Exchange Rates," Sustainability, MDPI, vol. 13(5), pages 1-29, March.
    11. Xinyi Li & Yinchuan Li & Hongyang Yang & Liuqing Yang & Xiao-Yang Liu, 2019. "DP-LSTM: Differential Privacy-inspired LSTM for Stock Prediction Using Financial News," Papers 1912.10806, arXiv.org.
    12. Berend Jelmer Dirk Gort & Xiao-Yang Liu & Xinghang Sun & Jiechao Gao & Shuaiyu Chen & Christina Dan Wang, 2022. "Deep Reinforcement Learning for Cryptocurrency Trading: Practical Approach to Address Backtest Overfitting," Papers 2209.05559, arXiv.org, revised Jan 2023.
    13. Ekaterina V. Orlova, 2023. "Dynamic Regimes for Corporate Human Capital Development Used Reinforcement Learning Methods," Mathematics, MDPI, vol. 11(18), pages 1-22, September.
    14. Amirhosein Mosavi & Yaser Faghan & Pedram Ghamisi & Puhong Duan & Sina Faizollahzadeh Ardabili & Ely Salwana & Shahab S. Band, 2020. "Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics," Mathematics, MDPI, vol. 8(10), pages 1-42, September.
    15. Xiao-Yang Liu & Hongyang Yang & Qian Chen & Runjia Zhang & Liuqing Yang & Bowen Xiao & Christina Dan Wang, 2020. "FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance," Papers 2011.09607, arXiv.org, revised Mar 2022.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xiao-Yang Liu & Hongyang Yang & Qian Chen & Runjia Zhang & Liuqing Yang & Bowen Xiao & Christina Dan Wang, 2020. "FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance," Papers 2011.09607, arXiv.org, revised Mar 2022.
    2. Amirhosein Mosavi & Yaser Faghan & Pedram Ghamisi & Puhong Duan & Sina Faizollahzadeh Ardabili & Ely Salwana & Shahab S. Band, 2020. "Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics," Mathematics, MDPI, vol. 8(10), pages 1-42, September.
    3. Ben Hambly & Renyuan Xu & Huining Yang, 2021. "Recent Advances in Reinforcement Learning in Finance," Papers 2112.04553, arXiv.org, revised Feb 2023.
    4. Schnaubelt, Matthias, 2022. "Deep reinforcement learning for the optimal placement of cryptocurrency limit orders," European Journal of Operational Research, Elsevier, vol. 296(3), pages 993-1006.
    5. Shuo Sun & Rundong Wang & Bo An, 2021. "Reinforcement Learning for Quantitative Trading," Papers 2109.13851, arXiv.org.
    6. Ayman Chaouki & Stephen Hardiman & Christian Schmidt & Emmanuel S'eri'e & Joachim de Lataillade, 2020. "Deep Deterministic Portfolio Optimization," Papers 2003.06497, arXiv.org, revised Apr 2020.
    7. Tidor-Vlad Pricope, 2021. "Deep Reinforcement Learning in Quantitative Algorithmic Trading: A Review," Papers 2106.00123, arXiv.org.
    8. Xiao-Yang Liu & Hongyang Yang & Jiechao Gao & Christina Dan Wang, 2021. "FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance," Papers 2111.09395, arXiv.org.
    9. Zechu Li & Xiao-Yang Liu & Jiahao Zheng & Zhaoran Wang & Anwar Walid & Jian Guo, 2021. "FinRL-Podracer: High Performance and Scalable Deep Reinforcement Learning for Quantitative Finance," Papers 2111.05188, arXiv.org.
    10. Stefan Mittnik & Nikolay Robinzonov & Klaus Wohlrabe, 2013. "The Micro Dynamics of Macro Announcements," CESifo Working Paper Series 4421, CESifo.
    11. Lepori, Gabriele M., 2015. "Investor mood and demand for stocks: Evidence from popular TV series finales," Journal of Economic Psychology, Elsevier, vol. 48(C), pages 33-47.
    12. Alessia Naccarato & Andrea Pierini & Giovanna Ferraro, 2021. "Markowitz portfolio optimization through pairs trading cointegrated strategy in long-term investment," Annals of Operations Research, Springer, vol. 299(1), pages 81-99, April.
    13. Tzuo Hann Law & Dongho Song & Amir Yaron, 2017. "Fearing the Fed: How Wall Street Reads Main Street," 2017 Meeting Papers 1632, Society for Economic Dynamics.
    14. Martin Širůček, 2013. "Impact of money supply on stock bubbles," Acta Universitatis Agriculturae et Silviculturae Mendelianae Brunensis, Mendel University Press, vol. 61(7), pages 2835-2842.
    15. Tulika Saha & Sriparna Saha & Pushpak Bhattacharyya, 2020. "Towards sentiment aided dialogue policy learning for multi-intent conversations using hierarchical reinforcement learning," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-28, July.
    16. Kari Harju & Syed Mujahid Hussain, 2011. "Intraday Seasonalities and Macroeconomic News Announcements," European Financial Management, European Financial Management Association, vol. 17(2), pages 367-390, March.
    17. Hartwell, Christopher A., 2014. "The impact of institutional volatility on financial volatility in transition economies : a GARCH family approach," BOFIT Discussion Papers 6/2014, Bank of Finland, Institute for Economies in Transition.
    18. Juan M. Londono & Mehrdad Samadi, 2023. "The Price of Macroeconomic Uncertainty: Evidence from Daily Options," International Finance Discussion Papers 1376, Board of Governors of the Federal Reserve System (U.S.).
    19. Mahmoud Mahfouz & Angelos Filos & Cyrine Chtourou & Joshua Lockhart & Samuel Assefa & Manuela Veloso & Danilo Mandic & Tucker Balch, 2019. "On the Importance of Opponent Modeling in Auction Markets," Papers 1911.12816, arXiv.org.
    20. Imen Azzouz & Wiem Fekih Hassen, 2023. "Optimization of Electric Vehicles Charging Scheduling Based on Deep Reinforcement Learning: A Decentralized Approach," Energies, MDPI, vol. 16(24), pages 1-18, December.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1907.01503. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.