IDEAS home Printed from
   My bibliography  Save this paper

Sparse Estimators and the Oracle Property, or the Return of Hodges' Estimator


  • Hannes Leeb

    (Dept. of Statistics, Yale University)

  • Benedikt M. Poetscher

    (Dept. of Statistics, University of Vienna)


We point out some pitfalls related to the concept of an oracle property as used in Fan and Li (2001, 2002, 2004) which are reminiscent of the well-known pitfalls related to Hodges’ estimator. The oracle property is often a consequence of sparsity of an estimator. We show that any estimator satisfying a sparsity property has maximal risk that converges to the supremum of the loss function; in particular, the maximal risk diverges to infinity when ever the loss function is unbounded. For ease of presentation the result is set in the framework of a linear regression model, but generalizes far beyond that setting. In a Monte Carlo study we also assess the extent of the problem infinite samples for the smoothly clipped absolute deviation (SCAD) estimator introduced in Fan and Li (2001). We find that this estimator can perform rather poorly infinite samples and that its worst-case performance relative to maximum likelihood deteriorates with increasing sample size when the estimator is tuned to sparsity.

Suggested Citation

  • Hannes Leeb & Benedikt M. Poetscher, 2005. "Sparse Estimators and the Oracle Property, or the Return of Hodges' Estimator," Cowles Foundation Discussion Papers 1500, Cowles Foundation for Research in Economics, Yale University, revised Apr 2007.
  • Handle: RePEc:cwl:cwldpp:1500

    Download full text from publisher

    File URL:
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    1. Zou, Hui, 2006. "The Adaptive Lasso and Its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 1418-1429, December.
    2. Bunea, Florentina & McKeague, Ian W., 2005. "Covariate selection for semiparametric hazard function regression models," Journal of Multivariate Analysis, Elsevier, vol. 92(1), pages 186-204, January.
    3. Leeb, Hannes & Pötscher, Benedikt M., 2005. "Model Selection And Inference: Facts And Fiction," Econometric Theory, Cambridge University Press, vol. 21(1), pages 21-59, February.
    4. Leeb, Hannes & Pötscher, Benedikt M., 2006. "Performance Limits For Estimators Of The Risk Or Distribution Of Shrinkage-Type Estimators, And Some General Lower Risk-Bound Results," Econometric Theory, Cambridge University Press, vol. 22(1), pages 69-97, February.
    5. Jianwen Cai & Jianqing Fan & Runze Li & Haibo Zhou, 2005. "Variable selection for multivariate failure time data," Biometrika, Biometrika Trust, vol. 92(2), pages 303-316, June.
    6. Kabaila, Paul, 1995. "The Effect of Model Selection on Confidence Regions and Prediction Regions," Econometric Theory, Cambridge University Press, vol. 11(3), pages 537-549, June.
    7. Jianqing Fan & Runze Li, 2004. "New Estimation and Model Selection Procedures for Semiparametric Modeling in Longitudinal Data Analysis," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 710-723, January.
    8. Pötscher, B.M., 1991. "Effects of Model Selection on Inference," Econometric Theory, Cambridge University Press, vol. 7(2), pages 163-185, June.
    9. Kabaila, Paul, 2002. "On Variable Selection In Linear Regression," Econometric Theory, Cambridge University Press, vol. 18(4), pages 913-925, August.
    10. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    Full references (including those not matched with items on IDEAS)

    More about this item


    Oracle property; Sparsity; Penalized maximum likelihood; Penalized least squares; Hodges’ estimator; SCAD; Lasso; Bridge estimator; Hard-thresholding; Maximal risk; Maximal absolute bias; Non-uniform limits;

    JEL classification:

    • C20 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - General
    • C51 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Construction and Estimation


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cwl:cwldpp:1500. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Matthew Regan). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.