Model Averaging with Ridge Regularization

Model averaging is an increasingly popular alternative to model selection. Ridge regression serves a similar purpose as model averaging, i.e. the minimization of mean squared error through shrinkage, though in different ways. In this paper, we propose the ridgeregularized modifications of Mallows model averaging (Hansen, 2007, Econometrica, 75) and heteroskedasticity-robust Mallows model averaging (Liu & Okui, 2013, The Econometrics Journal, 16) to leverage the capabilities of averaging and ridge regularization simultaneously. Via a simulation study, we examine the finite-sample improvements obtained by replacing least-squares with a ridge regression. Ridge-based model averaging is especially useful when one deals with sets of moderately to highly correlated predictors because the underlying ridge regression accommodates correlated predictors without blowing up estimation variance. A toy theoretical example shows that the relative reduction of mean squared error is increasing with the strength of the correlation. We also demonstrate the superiority of the ridge-regularized modifications via empirical examples focused on wages and economic growth.