IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2209.10458.html
   My bibliography  Save this paper

Model-Free Reinforcement Learning for Asset Allocation

Author

Listed:
  • Adebayo Oshingbesan
  • Eniola Ajiboye
  • Peruth Kamashazi
  • Timothy Mbaka

Abstract

Asset allocation (or portfolio management) is the task of determining how to optimally allocate funds of a finite budget into a range of financial instruments/assets such as stocks. This study investigated the performance of reinforcement learning (RL) when applied to portfolio management using model-free deep RL agents. We trained several RL agents on real-world stock prices to learn how to perform asset allocation. We compared the performance of these RL agents against some baseline agents. We also compared the RL agents among themselves to understand which classes of agents performed better. From our analysis, RL agents can perform the task of portfolio management since they significantly outperformed two of the baseline agents (random allocation and uniform allocation). Four RL agents (A2C, SAC, PPO, and TRPO) outperformed the best baseline, MPT, overall. This shows the abilities of RL agents to uncover more profitable trading strategies. Furthermore, there were no significant performance differences between value-based and policy-based RL agents. Actor-critic agents performed better than other types of agents. Also, on-policy agents performed better than off-policy agents because they are better at policy evaluation and sample efficiency is not a significant problem in portfolio management. This study shows that RL agents can substantially improve asset allocation since they outperform strong baselines. On-policy, actor-critic RL agents showed the most promise based on our analysis.

Suggested Citation

  • Adebayo Oshingbesan & Eniola Ajiboye & Peruth Kamashazi & Timothy Mbaka, 2022. "Model-Free Reinforcement Learning for Asset Allocation," Papers 2209.10458, arXiv.org.
    • Handle: RePEc:arx:papers:2209.10458
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2209.10458
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Yang Bao & Bin Ke & Bin Li & Y. Julia Yu & Jie Zhang, 2020. "Detecting Accounting Fraud in Publicly Traded U.S. Firms Using a Machine Learning Approach," Journal of Accounting Research, Wiley Blackwell, vol. 58(1), pages 199-235, March.
    2. Colombo, Emilio & Pelagatti, Matteo, 2020. "Statistical learning and exchange rate forecasting," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1260-1289.
    3. Stefania Albanesi & Domonkos F. Vamossy, 2019. "Predicting Consumer Default: A Deep Learning Approach," Working Papers 2019-056, Human Capital and Economic Opportunity Working Group.
    4. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," Review of Finance, European Finance Association, vol. 33(5), pages 2223-2273.
    5. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2223-2273.
    6. Salim Lahmiri & Stelios Bekiros, 2019. "Can machine learning approaches predict corporate bankruptcy? Evidence from a qualitative experimental design," Quantitative Finance, Taylor & Francis Journals, vol. 19(9), pages 1569-1577, September.
    7. Werner Antweiler & Murray Z. Frank, 2004. "Is All That Talk Just Noise? The Information Content of Internet Stock Message Boards," Journal of Finance, American Finance Association, vol. 59(3), pages 1259-1294, June.
    8. Wei Bao & Jun Yue & Yulei Rao, 2017. "A deep learning framework for financial time series using stacked autoencoders and long-short term memory," PLOS ONE, Public Library of Science, vol. 12(7), pages 1-24, July.
    9. Vamossy, Domonkos F., 2021. "Investor emotions and earnings announcements," Journal of Behavioral and Experimental Finance, Elsevier, vol. 30(C).
    10. Zhipeng Liang & Hao Chen & Junhao Zhu & Kangkang Jiang & Yanran Li, 2018. "Adversarial Deep Reinforcement Learning in Portfolio Management," Papers 1808.09940, arXiv.org, revised Nov 2018.
    11. Ahmet Murat Ozbayoglu & Mehmet Ugur Gudelek & Omer Berat Sezer, 2020. "Deep Learning for Financial Applications : A Survey," Papers 2002.05786, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hoang, Daniel & Wiegratz, Kevin, 2022. "Machine learning methods in finance: Recent applications and prospects," Working Paper Series in Economics 158, Karlsruhe Institute of Technology (KIT), Department of Economics and Management.
    2. Shuo Sun & Rundong Wang & Bo An, 2021. "Reinforcement Learning for Quantitative Trading," Papers 2109.13851, arXiv.org.
    3. Nawaf Almaskati, 2022. "Machine learning in finance: Major applications, issues, metrics, and future trends," International Journal of Financial Engineering (IJFE), World Scientific Publishing Co. Pte. Ltd., vol. 9(03), pages 1-32, September.
    4. Philippe Goulet Coulombe & Maxime Leroux & Dalibor Stevanovic & Stéphane Surprenant, 2022. "How is machine learning useful for macroeconomic forecasting?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 920-964, August.
    5. Avramov, Doron & Li, Minwen & Wang, Hao, 2021. "Predicting corporate policies using downside risk: A machine learning approach," Journal of Empirical Finance, Elsevier, vol. 63(C), pages 1-26.
    6. Colak, Gonul & Fu, Mengchuan & Hasan, Iftekhar, 2022. "On modeling IPO failure risk," Economic Modelling, Elsevier, vol. 109(C).
    7. Sergio Consoli & Luca Tiozzo Pezzoli & Elisa Tosetti, 2022. "Neural forecasting of the Italian sovereign bond market with economic news," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(S2), pages 197-224, December.
    8. Goodell, John W. & Kumar, Satish & Lim, Weng Marc & Pattnaik, Debidutta, 2021. "Artificial intelligence and machine learning in finance: Identifying foundations, themes, and research clusters from bibliometric analysis," Journal of Behavioral and Experimental Finance, Elsevier, vol. 32(C).
    9. Paul Geertsema & Helen Lu, 2023. "Relative Valuation with Machine Learning," Journal of Accounting Research, Wiley Blackwell, vol. 61(1), pages 329-376, March.
    10. Andrés Alonso Robisco & José Manuel Carbó Martínez, 2022. "Measuring the model risk-adjusted performance of machine learning algorithms in credit default prediction," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 8(1), pages 1-35, December.
    11. Domonkos F. Vamossy, 2023. "Social Media Emotions and IPO Returns," Papers 2306.12602, arXiv.org, revised Apr 2024.
    12. Nestoras Chalkidis & Rahul Savani, 2021. "Trading via Selective Classification," Papers 2110.14914, arXiv.org, revised Oct 2021.
    13. Zongwu Cai & Pixiong Chen, 2022. "New Online Investor Sentiment and Asset Returns," WORKING PAPERS SERIES IN THEORETICAL AND APPLIED ECONOMICS 202216, University of Kansas, Department of Economics, revised Nov 2022.
    14. Na, Haejung & Kim, Soonho, 2021. "Predicting stock prices based on informed traders’ activities using deep neural networks," Economics Letters, Elsevier, vol. 204(C).
    15. Hanauer, Matthias X. & Kononova, Marina & Rapp, Marc Steffen, 2022. "Boosting agnostic fundamental analysis: Using machine learning to identify mispricing in European stock markets," Finance Research Letters, Elsevier, vol. 48(C).
    16. Ngo, Vu Minh & Nguyen, Huan Huu & Van Nguyen, Phuc, 2023. "Does reinforcement learning outperform deep learning and traditional portfolio optimization models in frontier and developed financial markets?," Research in International Business and Finance, Elsevier, vol. 65(C).
    17. Xi Chen & Yang Ha (Tony) Cho & Yiwei Dou & Baruch Lev, 2022. "Predicting Future Earnings Changes Using Machine Learning and Detailed Financial Data," Journal of Accounting Research, Wiley Blackwell, vol. 60(2), pages 467-515, May.
    18. Yao, Haixiang & Xia, Shenghao & Liu, Hao, 2022. "Six-factor asset pricing and portfolio investment via deep learning: Evidence from Chinese stock market," Pacific-Basin Finance Journal, Elsevier, vol. 76(C).
    19. Luca Barbaglia & Sebastiano Manzan & Elisa Tosetti, 2023. "Forecasting Loan Default in Europe with Machine Learning," Journal of Financial Econometrics, Oxford University Press, vol. 21(2), pages 569-596.
    20. Zhao, Qi & Xu, Weijun & Ji, Yucheng, 2023. "Predicting financial distress of Chinese listed companies using machine learning: To what extent does textual disclosure matter?," International Review of Financial Analysis, Elsevier, vol. 89(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2209.10458. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.