An enhanced investor sentiment index

We propose an enhancement to the well-known market-based investor sentiment index by Baker, M., and J. Wurgler. (2006. “Investor Sentiment and the Cross-Section of Stock Returns.” The Journal of Finance 61 (4): 1645–1680). The low forecasting power of that index for future aggregate stock market returns has long been a puzzle, and we demonstrate that its ability to empirically capture latent investor sentiment can be significantly improved by allowing contributions of its individual components to vary over time, rather than being fixed. Our enhanced index represents an improved measure of investor sentiment: empirically, we find that our sentiment index not only instils forecasting power to the original Baker and Wurgler (2006) index, it also outperforms competitor measures in empirically capturing unobservable sentiment. This superior empirical performance is demonstrated to be due to investors’ irrational expectations about future discount rates. Lastly, sentiment measured by our enhanced index contains useful and unique information about future market returns, as it outperforms fundamental economic predictors in forecasting, in both the statistical and economic sense.