Machine Learning Forecasting of U.S. Stock Market Volatility: The Role of Stock and Oil Bubbles

This study examines the predictive power of multi-scale positive and negative speculative bubbles in equity and energy markets for S&P 500 realized variance across horizons from 1 to 24 months. Using a hierarchical modeling framework and machine learning estimators, the analysis evaluates whether stock and oil bubbles provide incremental information beyond macroeconomic variables and financial uncertainty. Applying Clark and West's (2007) tests for nested model comparisons, the results reveal a hierarchy in predictive content that varies by forecast horizon. At the 1-month horizon, neither stock nor oil bubbles improves forecast accuracy. At the 3-month horizon, oil bubbles emerge as the dominant predictor; the Bayesian Regularized Neural Network (BRNN) estimator achieves a statistically significant improvement when oil bubbles are included with stock bubbles, resulting in a 30.7 percent reduction in mean squared error (MSE). At the 6-month horizon, stock bubbles become more important, with both the Gradient Boosting Machine (GBM) and BRNN estimators showing significant improvements. For longer horizons, oil bubbles remain relevant, but their predictive value depends on the estimator: BRNN captures oil bubble effects at 12 months, while GBM does so at 24 months. These findings highlight the importance of horizonspecific model selection and indicate a complex transmission of speculative shocks across asset classes.