Inference for High-Dimensional Model Averaging Estimators

Selection methods for high-dimensional models are well developed, but they do not take into account the choice of the model, which leads to an underestimation of the variability of the estimator. We propose a procedure for model averaging in high-dimensional regression models that allows inference even when the number of predictors is larger than the sample size. The proposed estimator is constructed from the debiased Lasso and the weights are chosen to reduce the prediction risk. We derive the asymptotic distribution of the estimator within a high-dimensional framework and offer guarantees for the minimal loss prediction obtained using our choice of the weights. In contrast to existing approaches, our proposedmethod combines the advantages of model averaging with the possibility of inference based on asymptotic normality. The estimator shows a smaller prediction risk than its competitors when applied to a real, high-dimensional dataset and along various simulation studies, confirming our theoretical results.