Pivotal estimation via square-root lasso in nonparametric regression
We propose a self-tuning ? Lasso method that simultaneiously resolves three important practical problems in high-dimensional regression analysis, namely it handles the unknown scale, heteroscedasticity, and (drastic) non-Gaussianity of the noise. In addition, our analysis allows for badly behaved designs, for example perfectly collinear regressors, and generates sharp bounds even in extreme cases, such as the infinite variance case and the noiseless case, in contrast to Lasso. We establish various non-asymptotic bounds for ? Lasso including prediction norm rate and sharp sparcity bound. Our analysis is based on new impact factors that are tailored to establish prediction rates. In order to cover heteroscedastic non-Gaussian noise, we rely on moderate deviation theory for self-normalized sums to achieve Gaussian-like results under weak conditions. Moreover, we derive bounds on the performance of ordinary least square (ols) applied to the model selected by ? Lasso accounting for possible misspecification of the selected model. Under mild conditions the rate of convergence of ols post ? Lasso is no worse than ? Lasso even with a misspecified selected model and possibly better otherwise. As an application, we consider the use of ? Lasso and post ? Lasso as estimators of nuisance parameters in a generic semi-parametric problem (nonlinear instrumental/moment condition or Z-estimation problem).
|Date of creation:||10 Dec 2013|
|Date of revision:|
|Contact details of provider:|| Postal: The Institute for Fiscal Studies 7 Ridgmount Street LONDON WC1E 7AE|
Phone: (+44) 020 7291 4800
Fax: (+44) 020 7323 4780
Web page: http://cemmap.ifs.org.uk
More information through EDIRC
|Order Information:|| Postal: The Institute for Fiscal Studies 7 Ridgmount Street LONDON WC1E 7AE|
References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Jianqing Fan & Jinchi Lv, 2008. "Sure independence screening for ultrahigh dimensional feature space," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(5), pages 849-911.
- Eric Gautier & Alexandre Tsybakov, 2014.
"High-dimensional instrumental variables regression and confidence sets,"
- Eric Gautier & Alexandre Tsybakov, 2011. "High-Dimensional Instrumental Variables Regression and Confidence Sets," Working Papers 2011-13, Centre de Recherche en Economie et Statistique.
- Amemiya, Takeshi, 1977. "The Maximum Likelihood and the Nonlinear Three-Stage Least Squares Estimator in the General Nonlinear Simultaneous Equation Model," Econometrica, Econometric Society, vol. 45(4), pages 955-68, May.
- Tingni Sun & Cun-Hui Zhang, 2012. "Scaled sparse linear regression," Biometrika, Biometrika Trust, vol. 99(4), pages 879-898.
- A. Belloni & D. Chen & V. Chernozhukov & C. Hansen, 2012.
"Sparse Models and Methods for Optimal Instruments With an Application to Eminent Domain,"
Econometric Society, vol. 80(6), pages 2369-2429, November.
- Alexandre Belloni & D. Chen & Victor Chernozhukov & Christian Hansen, 2010. "Sparse models and methods for optimal instruments with an application to eminent domain," CeMMAP working papers CWP31/10, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
- A. Belloni & V. Chernozhukov & L. Wang, 2011. "Square-root lasso: pivotal recovery of sparse signals via conic programming," Biometrika, Biometrika Trust, vol. 98(4), pages 791-806.
- Hans-Ulrich Derlien & B. Guy Peters, 2008. "Introduction," Chapters, in: The State at Work, Volume 2, chapter 1 Edward Elgar Publishing.
- Hansen, Lars Peter, 1982. "Large Sample Properties of Generalized Method of Moments Estimators," Econometrica, Econometric Society, vol. 50(4), pages 1029-54, July.
When requesting a correction, please mention this item's handle: RePEc:ifs:cemmap:62/13. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Emma Hyman)
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If references are entirely missing, you can add them using this form.
If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.
Please note that corrections may take a couple of weeks to filter through the various RePEc services.