Sparse Estimators and the Oracle Property, or the Return of Hodges' Estimator
We point out some pitfalls related to the concept of an oracle property as used in Fan and Li (2001, 2002, 2004) which are reminiscent of the well-known pitfalls related to Hodges’ estimator. The oracle property is often a consequence of sparsity of an estimator. We show that any estimator satisfying a sparsity property has maximal risk that converges to the supremum of the loss function; in particular, the maximal risk diverges to infinity when ever the loss function is unbounded. For ease of presentation the result is set in the framework of a linear regression model, but generalizes far beyond that setting. In a Monte Carlo study we also assess the extent of the problem infinite samples for the smoothly clipped absolute deviation (SCAD) estimator introduced in Fan and Li (2001). We find that this estimator can perform rather poorly infinite samples and that its worst-case performance relative to maximum likelihood deteriorates with increasing sample size when the estimator is tuned to sparsity.
|Date of creation:||Feb 2005|
|Date of revision:||Apr 2007|
|Publication status:||Published in Journal of Econometrics (2008), 142: 201-211|
|Contact details of provider:|| Postal: Yale University, Box 208281, New Haven, CT 06520-8281 USA|
Phone: (203) 432-3702
Fax: (203) 432-6167
Web page: http://cowles.yale.edu/
More information through EDIRC
|Order Information:|| Postal: Cowles Foundation, Yale University, Box 208281, New Haven, CT 06520-8281 USA|
References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Kabaila, Paul, 2002. "On Variable Selection In Linear Regression," Econometric Theory, Cambridge University Press, vol. 18(04), pages 913-925, August.
- Hannes Leeb & Benedikt M. Pötscher, 2003.
"Performance Limits for Estimators of the Risk or Distribution of Shrinkage-Type Estimators, and Some General Lower Risk-Bound Results,"
Vienna Economics Papers
0301, University of Vienna, Department of Economics.
- Leeb, Hannes & P tscher, Benedikt M., 2006. "Performance Limits For Estimators Of The Risk Or Distribution Of Shrinkage-Type Estimators, And Some General Lower Risk-Bound Results," Econometric Theory, Cambridge University Press, vol. 22(01), pages 69-97, February.
- Leeb, Hannes & P tscher, Benedikt M., 2005. "Model Selection And Inference: Facts And Fiction," Econometric Theory, Cambridge University Press, vol. 21(01), pages 21-59, February.
- Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
- Kabaila, Paul, 1995. "The Effect of Model Selection on Confidence Regions and Prediction Regions," Econometric Theory, Cambridge University Press, vol. 11(03), pages 537-549, June.
- Jianwen Cai & Jianqing Fan & Runze Li & Haibo Zhou, 2005. "Variable selection for multivariate failure time data," Biometrika, Biometrika Trust, vol. 92(2), pages 303-316, June.
- Pötscher, B.M., 1991. "Effects of Model Selection on Inference," Econometric Theory, Cambridge University Press, vol. 7(02), pages 163-185, June.
- Jianqing Fan & Runze Li, 2004. "New Estimation and Model Selection Procedures for Semiparametric Modeling in Longitudinal Data Analysis," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 710-723, January.
- Zou, Hui, 2006. "The Adaptive Lasso and Its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 1418-1429, December.
- Bunea, Florentina & McKeague, Ian W., 2005. "Covariate selection for semiparametric hazard function regression models," Journal of Multivariate Analysis, Elsevier, vol. 92(1), pages 186-204, January.
When requesting a correction, please mention this item's handle: RePEc:cwl:cwldpp:1500. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Matthew C. Regan)
If references are entirely missing, you can add them using this form.