Predicting Stock Returns Using Machine Learning: A Hybrid Approach with LightGBM, XGBoost, and Portfolio Optimization

Predicting stock returns presents a multifaceted challenge in the financial market, as it is influenced by a myriad of variables, including economic indicators, market sentiment, and company performance. Traditional predictive models often fall short in capturing the complexities and dynamics of contemporary markets. As a result, there has been a notable shift towards machine learning approaches, which offer advanced techniques for analyzing vast datasets. This study explores the application of two leading machine-learning models, LightGBM and XGBoost, in forecasting the future performance of S&P 500 stocks. By integrating both fundamental and technical data—such as price trends, financial metrics, and broader market conditions—these models aim to deliver accurate predictions. The insights generated can significantly inform investment strategies, enabling investors to identify favorable long and short-term positions, thereby optimizing their portfolio performance in a highly competitive and rapidly changing financial landscape. This research contributes to the evolving field of quantitative finance, highlighting the potential of machine learning in investment decision-making.