Pac-Bayesian Bounds for Sparse Regression Estimation with Exponential Weights
AbstractWe consider the sparse regression model where the number of parametersp is larger than the sample size n. The difficulty when consideringhigh-dimensional problems is to propose estimators achieving a good compromisebetween statistical and computational performances. The BIC estimatorfor instance performs well from the statistical point of view  but can onlybe computed for values of p of at most a few tens. The Lasso estimator issolution of a convex minimization problem, hence computable for large valueof p. However stringent conditions on the design are required to establish fastrates of convergence for this estimator. Dalalyan and Tsybakov  proposea method achieving a good compromise between the statistical and computationalaspects of the problem. Their estimator can be computed for reasonablylarge p and satisfies nice statistical properties under weak assumptions on thedesign. However,  proposes sparsity oracle inequalities in expectation forthe empirical excess risk only. In this paper, we propose an aggregation proceduresimilar to that of  but with improved statistical performances. Ourmain theoretical result is a sparsity oracle inequality in probability for the trueexcess risk for a version of exponential weight estimator. We also propose aMCMC method to compute our estimator for reasonably large values of p.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoPaper provided by Centre de Recherche en Economie et Statistique in its series Working Papers with number 2010-40.
Date of creation: 2010
Date of revision:
You can help add them by filling out this form.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Florian Sallaberry).
If references are entirely missing, you can add them using this form.