Robust Bayesian high-dimensional variable selection and inference with the horseshoe family of priors

Frequentist robust variable selection has been extensively investigated in high-dimensional regression. Despite success, developing the corresponding statistical inference procedures remains a challenging task. Recently, tackling this challenge from a Bayesian perspective has received much attention. In literature, the two-group spike-and-slab priors that can induce exact sparsity have been demonstrated to yield valid inference in robust sparse linear models. Nevertheless, another important category of sparse priors, the horseshoe family of priors, including horseshoe, horseshoe+, and regularized horseshoe priors, has not yet been examined in robust high-dimensional regression by far. Their performance in variable selection and especially statistical inference in the presence of heavy-tailed model errors is not well understood. To address this gap, robust Bayesian hierarchical models incorporating the horseshoe family of priors are developed, together with an efficient Gibbs sampling scheme for posterior computation. Numerical studies demonstrate that, compared with competing methods employing alternative sampling strategies such as slice sampling, the proposed methods lead to superior performance in variable selection, Bayesian estimation and statistical inference. In particular, even without imposing exact sparsity, the one-group horseshoe priors can still yield valid Bayesian credible intervals under robust high-dimensional linear regression models. Applications to real data further demonstrate the advantages of the proposed methods over competitors.