IDEAS home Printed from https://ideas.repec.org/a/eee/pacfin/v91y2025ics0927538x25000836.html
   My bibliography  Save this article

Optimization-based spectral end-to-end deep reinforcement learning for equity portfolio management

Author

Listed:
  • Yu, Pengrui
  • Liu, Siya
  • Jin, Chengneng
  • Gu, Runsheng
  • Gong, Xiaomin

Abstract

We propose a novel approach to equity portfolio optimization that combines spectral analysis and classical equity portfolio optimization theory with deep reinforcement learning in an end-to-end framework. We introduce the End-to-end Frequency Online Deep Deterministic Policy Gradient (EFO-DDPG) algorithm, which leverages discrete Fourier transform to decompose asset return sequences into frequency components. Unlike traditional methods that treat high-frequency components as noise, EFO-DDPG learns to adjust the influence of different frequency components dynamically. Moreover, the algorithm embeds a mean–variance portfolio optimization problem within a deep learning network, enhancing interpretability compared to black-box approaches. The framework models the investment problem as a Partially Observable Markov Decision Process (POMDP), using a state processing block with transformer encoders to capture complex relationships in the market data. By integrating spectral analysis, portfolio optimization theory, and online deep reinforcement learning, EFO-DDPG aims to adapt to non-stationary financial markets and generate superior investment strategies.

Suggested Citation

  • Yu, Pengrui & Liu, Siya & Jin, Chengneng & Gu, Runsheng & Gong, Xiaomin, 2025. "Optimization-based spectral end-to-end deep reinforcement learning for equity portfolio management," Pacific-Basin Finance Journal, Elsevier, vol. 91(C).
  • Handle: RePEc:eee:pacfin:v:91:y:2025:i:c:s0927538x25000836
    DOI: 10.1016/j.pacfin.2025.102746
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0927538X25000836
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.pacfin.2025.102746?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Victor DeMiguel & Lorenzo Garlappi & Raman Uppal, 2009. "Optimal Versus Naive Diversification: How Inefficient is the 1-N Portfolio Strategy?," The Review of Financial Studies, Society for Financial Studies, vol. 22(5), pages 1915-1953, May.
    2. Klein, Roger W. & Bawa, Vijay S., 1976. "The effect of estimation risk on optimal portfolio choice," Journal of Financial Economics, Elsevier, vol. 3(3), pages 215-231, June.
    3. Bandi, Federico M. & Chaudhuri, Shomesh E. & Lo, Andrew W. & Tamoni, Andrea, 2021. "Spectral factor models," Journal of Financial Economics, Elsevier, vol. 142(1), pages 214-238.
    4. A. Sinem Uysal & Xiaoyue Li & John M. Mulvey, 2024. "End-to-end risk budgeting portfolio optimization with neural networks," Annals of Operations Research, Springer, vol. 339(1), pages 397-426, August.
    5. Zhengyao Jiang & Dixing Xu & Jinjun Liang, 2017. "A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem," Papers 1706.10059, arXiv.org, revised Jul 2017.
    6. Zhipeng Liang & Hao Chen & Junhao Zhu & Kangkang Jiang & Yanran Li, 2018. "Adversarial Deep Reinforcement Learning in Portfolio Management," Papers 1808.09940, arXiv.org, revised Nov 2018.
    7. Zihao Zhang & Stefan Zohren & Stephen Roberts, 2019. "Deep Reinforcement Learning for Trading," Papers 1911.10107, arXiv.org.
    8. Harry Markowitz, 1952. "Portfolio Selection," Journal of Finance, American Finance Association, vol. 7(1), pages 77-91, March.
    9. Andrew Ang & Allan Timmermann, 2012. "Regime Changes and Financial Markets," Annual Review of Financial Economics, Annual Reviews, vol. 4(1), pages 313-337, October.
    10. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2223-2273.
    11. Le Trung Hieu, 2020. "Deep Reinforcement Learning for Stock Portfolio Optimization," Papers 2012.06325, arXiv.org.
    12. Marianne Baxter & Robert G. King, 1999. "Measuring Business Cycles: Approximate Band-Pass Filters For Economic Time Series," The Review of Economics and Statistics, MIT Press, vol. 81(4), pages 575-593, November.
    13. Mark Britten‐Jones, 1999. "The Sampling Error in Estimates of Mean‐Variance Efficient Portfolio Weights," Journal of Finance, American Finance Association, vol. 54(2), pages 655-671, April.
    14. Bruno Scalzo & Alvaro Arroyo & Ljubisa Stankovic & Danilo P. Mandic, 2021. "Nonstationary Portfolios: Diversification in the Spectral Domain," Papers 2102.00477, arXiv.org.
    15. Christophe Croux & Mario Forni & Lucrezia Reichlin, 2001. "A Measure Of Comovement For Economic Variables: Theory And Empirics," The Review of Economics and Statistics, MIT Press, vol. 83(2), pages 232-241, May.
    16. Jun, Doobae & Ahn, Changmo & Kim, Jinsu & Kim, Gwangil, 2019. "Signal analysis of global financial crises using Fourier series," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 526(C).
    17. McNevin, Bruce D. & Nix, Joan, 2018. "The beta heuristic from a time/frequency perspective: A wavelet analysis of the market risk of sectors," Economic Modelling, Elsevier, vol. 68(C), pages 570-585.
    18. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," Review of Finance, European Finance Association, vol. 33(5), pages 2223-2273.
    19. Zikai Wei & Bo Dai & Dahua Lin, 2023. "E2EAI: End-to-End Deep Learning Framework for Active Investing," Papers 2305.16364, arXiv.org.
    20. Ramsey, J.B., 2002. "Wavelets in Economics and Finance: Past and Future," Working Papers 02-02, C.V. Starr Center for Applied Economics, New York University.
    21. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    22. Chao Zhang & Zihao Zhang & Mihai Cucuringu & Stefan Zohren, 2021. "A Universal End-to-End Approach to Portfolio Optimization via Deep Learning," Papers 2111.09170, arXiv.org.
    23. Giorgio Costa & Garud N. Iyengar, 2023. "Distributionally robust end-to-end portfolio construction," Quantitative Finance, Taylor & Francis Journals, vol. 23(10), pages 1465-1482, October.
    24. Norden E. Huang & Man‐Li Wu & Wendong Qu & Steven R. Long & Samuel S. P. Shen, 2003. "Applications of Hilbert–Huang transform to non‐stationary financial time series analysis," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 19(3), pages 245-268, July.
    25. R. Cont, 2001. "Empirical properties of asset returns: stylized facts and statistical issues," Quantitative Finance, Taylor & Francis Journals, vol. 1(2), pages 223-236.
    26. Andrew Butler & Roy H. Kwon, 2023. "Integrating prediction in mean-variance portfolio optimization," Quantitative Finance, Taylor & Francis Journals, vol. 23(3), pages 429-452, March.
    27. Yunan Ye & Hengzhi Pei & Boxin Wang & Pin-Yu Chen & Yada Zhu & Jun Xiao & Bo Li, 2020. "Reinforcement-Learning based Portfolio Management with Augmented Asset Movement Prediction States," Papers 2002.05780, arXiv.org.
    28. Fischer, Thomas G., 2018. "Reinforcement learning in financial markets - a survey," FAU Discussion Papers in Economics 12/2018, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    29. Ramsey James B., 2002. "Wavelets in Economics and Finance: Past and Future," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 6(3), pages 1-29, November.
    30. Xiao-Yang Liu & Hongyang Yang & Qian Chen & Runjia Zhang & Liuqing Yang & Bowen Xiao & Christina Dan Wang, 2020. "FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance," Papers 2011.09607, arXiv.org, revised Mar 2022.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shuo Sun & Rundong Wang & Bo An, 2021. "Reinforcement Learning for Quantitative Trading," Papers 2109.13851, arXiv.org.
    2. Ben Hambly & Renyuan Xu & Huining Yang, 2021. "Recent Advances in Reinforcement Learning in Finance," Papers 2112.04553, arXiv.org, revised Feb 2023.
    3. Ngo, Vu Minh & Nguyen, Huan Huu & Van Nguyen, Phuc, 2023. "Does reinforcement learning outperform deep learning and traditional portfolio optimization models in frontier and developed financial markets?," Research in International Business and Finance, Elsevier, vol. 65(C).
    4. Caldeira, João F. & Santos, André A.P. & Torrent, Hudson S., 2023. "Semiparametric portfolios: Improving portfolio performance by exploiting non-linearities in firm characteristics," Economic Modelling, Elsevier, vol. 122(C).
    5. Bruno Scalzo & Alvaro Arroyo & Ljubisa Stankovic & Danilo P. Mandic, 2021. "Nonstationary Portfolios: Diversification in the Spectral Domain," Papers 2102.00477, arXiv.org.
    6. Conlon, Thomas & Cotter, John & Kynigakis, Iason, 2025. "Asset allocation with factor-based covariance matrices," European Journal of Operational Research, Elsevier, vol. 325(1), pages 189-203.
    7. Shomesh E. Chaudhuri & Andrew W. Lo, 2019. "Dynamic Alpha: A Spectral Decomposition of Investment Performance Across Time Horizons," Management Science, INFORMS, vol. 65(9), pages 4440-4450, September.
    8. Tu, Xueyong & Li, Bin, 2024. "Robust portfolio selection with smart return prediction," Economic Modelling, Elsevier, vol. 135(C).
    9. Yilie Huang & Yanwei Jia & Xun Yu Zhou, 2024. "Mean--Variance Portfolio Selection by Continuous-Time Reinforcement Learning: Algorithms, Regret Analysis, and Empirical Study," Papers 2412.16175, arXiv.org, revised Aug 2025.
    10. Kubo, Kenji & Nakagawa, Kei, 2025. "Portfolio optimization using deep learning with risk aversion utility function," Finance Research Letters, Elsevier, vol. 74(C).
    11. Eric Benhamou & David Saltiel & Sandrine Ungari & Abhishek Mukhopadhyay, 2020. "Bridging the gap between Markowitz planning and deep reinforcement learning," Papers 2010.09108, arXiv.org.
    12. Cong, Lin William & Feng, Guanhao & He, Jingyu & He, Xin, 2025. "Growing the efficient frontier on panel trees," Journal of Financial Economics, Elsevier, vol. 167(C).
    13. Brini, Alessio & Tantari, Daniele, 2023. "Deep reinforcement trading with predictable returns," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 622(C).
    14. Immo Stadtmüller & Benjamin R. Auer & Frank Schuhmacher, 2024. "Core-satellite investing with commodity futures momentum," Journal of Asset Management, Palgrave Macmillan, vol. 25(3), pages 261-287, May.
    15. Andreas Koukorinis & Gareth W. Peters & Guido Germano, 2025. "Generative-Discriminative Machine Learning Models for High-Frequency Financial Regime Classification," Methodology and Computing in Applied Probability, Springer, vol. 27(2), pages 1-32, June.
    16. Wang, Jianzhou & Lv, Mengzheng & Wang, Shuai & Gao, Jialu & Zhao, Yang & Wang, Qiangqiang, 2024. "Can multi-period auto-portfolio systems improve returns? Evidence from Chinese and U.S. stock markets," International Review of Financial Analysis, Elsevier, vol. 95(PB).
    17. Sun, Chuting & Wu, Qi & Yan, Xing, 2024. "Dynamic CVaR portfolio construction with attention-powered generative factor learning," Journal of Economic Dynamics and Control, Elsevier, vol. 160(C).
    18. Ni, Xuanming & Zheng, Tiantian & Zhao, Huimin & Zhu, Shushang, 2023. "High-dimensional portfolio optimization based on tree-structured factor model," Pacific-Basin Finance Journal, Elsevier, vol. 81(C).
    19. Ko, Hyungjin & Son, Bumho & Lee, Jaewook, 2024. "A novel integration of the Fama–French and Black–Litterman models to enhance portfolio management," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 91(C).
    20. Kim Hiang Liow & Xiaoxia Zhou & Qiang Li & Yuting Huang, 2019. "Co-movement between the US and the securitised real estate markets of the Asian-Pacific economies," Journal of Property Research, Taylor & Francis Journals, vol. 36(1), pages 27-58, January.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:pacfin:v:91:y:2025:i:c:s0927538x25000836. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/pacfin .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.