Optimizing stock portfolios with deep reinforcement learning after FNN-based fundamental analysis

Traditional stock selection and optimization strategies are significantly restricted by the unpredictable nature of time-varying financial markets. However, flexible machine learning (ML) models can adapt to environmental changes and make timely investment decisions. In this study, we employ fundamental analyses using feedforward neural networks (FNN) to select stocks and propose a new deep reinforcement learning (DRL) method to further adjust allocations. To our knowledge, this is the first ML model to offer complete portfolio management functionalities with financial interpretability based on fundamental analysis. The reward function of our DRL model is based on the relative return to an equal-weighted (EW) portfolio, ensuring the stability of trading policy learning. We construct the policy network using temporal convolutional networks (TCN) that incorporate concatenated weekly (biweekly) aggregated information to learn stock price time series. Our model employs proximal policy optimization (PPO) to select allocation strategies, minimizing the impact of significant position changes and associated transaction costs. The resulting model outperforms traditional convolutional neural networks (CNN) and other commonly used policy gradient models. Evaluation using S&P 100 index data from 2009 to 2021 demonstrates that each component and combination significantly enhances portfolio performance.