Advanced Search
MyIDEAS: Login

Sparse Estimators and the Oracle Property, or the Return of Hodges' Estimator

Contents:

Author Info

  • Hannes Leeb

    (Dept. of Statistics, Yale University)

  • Benedikt M. Poetscher

    (Dept. of Statistics, University of Vienna)

Abstract

We point out some pitfalls related to the concept of an oracle property as used in Fan and Li (2001, 2002, 2004) which are reminiscent of the well-known pitfalls related to Hodges’ estimator. The oracle property is often a consequence of sparsity of an estimator. We show that any estimator satisfying a sparsity property has maximal risk that converges to the supremum of the loss function; in particular, the maximal risk diverges to infinity when ever the loss function is unbounded. For ease of presentation the result is set in the framework of a linear regression model, but generalizes far beyond that setting. In a Monte Carlo study we also assess the extent of the problem infinite samples for the smoothly clipped absolute deviation (SCAD) estimator introduced in Fan and Li (2001). We find that this estimator can perform rather poorly infinite samples and that its worst-case performance relative to maximum likelihood deteriorates with increasing sample size when the estimator is tuned to sparsity.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://cowles.econ.yale.edu/P/cd/d15a/d1500.pdf
Download Restriction: no

Bibliographic Info

Paper provided by Cowles Foundation for Research in Economics, Yale University in its series Cowles Foundation Discussion Papers with number 1500.

as in new window
Length: 18 pages
Date of creation: Feb 2005
Date of revision: Apr 2007
Publication status: Published in Journal of Econometrics (2008), 142: 201-211
Handle: RePEc:cwl:cwldpp:1500

Contact details of provider:
Postal: Yale University, Box 208281, New Haven, CT 06520-8281 USA
Phone: (203) 432-3702
Fax: (203) 432-6167
Web page: http://cowles.econ.yale.edu/
More information through EDIRC

Order Information:
Postal: Cowles Foundation, Yale University, Box 208281, New Haven, CT 06520-8281 USA

Related research

Keywords: Oracle property; Sparsity; Penalized maximum likelihood; Penalized least squares; Hodges’ estimator; SCAD; Lasso; Bridge estimator; Hard-thresholding; Maximal risk; Maximal absolute bias; Non-uniform limits;

Other versions of this item:

Find related papers by JEL classification:

This paper has been announced in the following NEP Reports:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Kabaila, Paul, 1995. "The Effect of Model Selection on Confidence Regions and Prediction Regions," Econometric Theory, Cambridge University Press, vol. 11(03), pages 537-549, June.
  2. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
  3. Leeb, Hannes & P tscher, Benedikt M., 2006. "Performance Limits For Estimators Of The Risk Or Distribution Of Shrinkage-Type Estimators, And Some General Lower Risk-Bound Results," Econometric Theory, Cambridge University Press, vol. 22(01), pages 69-97, February.
  4. Bunea, Florentina & McKeague, Ian W., 2005. "Covariate selection for semiparametric hazard function regression models," Journal of Multivariate Analysis, Elsevier, vol. 92(1), pages 186-204, January.
  5. Jianqing Fan & Runze Li, 2004. "New Estimation and Model Selection Procedures for Semiparametric Modeling in Longitudinal Data Analysis," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 710-723, January.
  6. Zou, Hui, 2006. "The Adaptive Lasso and Its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 1418-1429, December.
  7. Kabaila, Paul, 2002. "On Variable Selection In Linear Regression," Econometric Theory, Cambridge University Press, vol. 18(04), pages 913-925, August.
  8. Jianwen Cai & Jianqing Fan & Runze Li & Haibo Zhou, 2005. "Variable selection for multivariate failure time data," Biometrika, Biometrika Trust, vol. 92(2), pages 303-316, June.
  9. Pötscher, B.M., 1991. "Effects of Model Selection on Inference," Econometric Theory, Cambridge University Press, vol. 7(02), pages 163-185, June.
  10. Leeb, Hannes & P tscher, Benedikt M., 2005. "Model Selection And Inference: Facts And Fiction," Econometric Theory, Cambridge University Press, vol. 21(01), pages 21-59, February.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. Xu Cheng & Zhipeng Liao, 2012. "Select the Valid and Relevant Moments: A One-Step Procedure for GMM with Many Moments," PIER Working Paper Archive 12-045, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
  2. Alexandre Belloni & Victor Chernozhukov & Kengo Kato, 2013. "Uniform post selection inference for LAD regression models," CeMMAP working papers CWP24/13, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
  3. Ulrike Schneider & Martin Wagner, 2012. "Catching Growth Determinants with the Adaptive Lasso," German Economic Review, Verein für Socialpolitik, vol. 13(1), pages 71-85, 02.
  4. Malene Kallestrup-Lamb & Anders Bredahl Kock & Johannes Tang Kristensen, 2013. "Lassoing the Determinants of Retirement," CREATES Research Papers 2013-21, School of Economics and Management, University of Aarhus.
  5. Jörg Stoye, 2008. "More on confidence intervals for partially identified parameters," CeMMAP working papers CWP11/08, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
  6. Pötscher, Benedikt M. & Schneider, Ulrike, 2011. "Distributional results for thresholding estimators in high-dimensional Gaussian regression models," MPRA Paper 31882, University Library of Munich, Germany.
  7. Eustasio Barrio, 2010. "Comments on: l 1 -penalization for mixture regression models," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer, vol. 19(2), pages 276-279, August.
  8. Matei Demetrescu & Uwe Hassler & Vladimir Kuzin, 2011. "Pitfalls of post-model-selection testing: experimental quantification," Empirical Economics, Springer, vol. 40(2), pages 359-372, April.
  9. Pötscher, Benedikt M. & Leeb, Hannes, 2007. "On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding," MPRA Paper 5615, University Library of Munich, Germany.
  10. Anders Bredahl Kock & Laurent A.F. Callot, 2012. "Oracle Efficient Estimation and Forecasting with the Adaptive LASSO and the Adaptive Group LASSO in Vector Autoregressions," CREATES Research Papers 2012-38, School of Economics and Management, University of Aarhus.
  11. Alexandre Belloni & Victor Chernozhukov & Kengo Kato, 2013. "Uniform post selection inference for LAD regression and other z-estimation problems," CeMMAP working papers CWP74/13, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
  12. Pötscher, Benedikt M., 2007. "Confidence Sets Based on Sparse Estimators Are Necessarily Large," MPRA Paper 5677, University Library of Munich, Germany.
  13. Pötscher, Benedikt M. & Schneider, Ulrike, 2008. "Confidence sets based on penalized maximum likelihood estimators," MPRA Paper 9062, University Library of Munich, Germany.
  14. Anders Bredahl Kock, 2012. "On the Oracle Property of the Adaptive Lasso in Stationary and Nonstationary Autoregressions," CREATES Research Papers 2012-05, School of Economics and Management, University of Aarhus.
  15. Alexandre Belloni & Victor Chernozhukov & Kengo Kato, 2013. "Robust inference in high-dimensional approximately sparse quantile regression models," CeMMAP working papers CWP70/13, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
  16. Pötscher, Benedikt M. & Schneider, Ulrike, 2007. "On the distribution of the adaptive LASSO estimator," MPRA Paper 6913, University Library of Munich, Germany.
  17. Zhipeng Liao & Peter C.B. Phillips, 2012. "Automated Estimation of Vector Error Correction Models," Cowles Foundation Discussion Papers 1873, Cowles Foundation for Research in Economics, Yale University.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:cwl:cwldpp:1500. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Glena Ames).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.