Advanced Search
MyIDEAS: Login

On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding

Contents:

Author Info

  • Pötscher, Benedikt M.
  • Leeb, Hannes

Abstract

We study the distributions of the LASSO, SCAD, and thresholding estimators, in finite samples and in the large-sample limit. The asymptotic distributions are derived for both the case where the estimators are tuned to perform consistent model selection and for the case where the estimators are tuned to perform conservative model selection. Our findings complement those of Knight and Fu (2000) and Fan and Li (2001). We show that the distributions are typically highly nonnormal regardless of how the estimator is tuned, and that this property persists in large samples. An impossibility result regarding estimation of the estimators' distribution function is also provided.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://mpra.ub.uni-muenchen.de/5615/
File Function: original version
Download Restriction: no

File URL: http://mpra.ub.uni-muenchen.de/14708/
File Function: revised version
Download Restriction: no

Bibliographic Info

Paper provided by University Library of Munich, Germany in its series MPRA Paper with number 5615.

as in new window
Length:
Date of creation: Oct 2007
Date of revision:
Handle: RePEc:pra:mprapa:5615

Contact details of provider:
Postal: Schackstr. 4, D-80539 Munich, Germany
Phone: +49-(0)89-2180-2219
Fax: +49-(0)89-2180-3900
Web page: http://mpra.ub.uni-muenchen.de
More information through EDIRC

Related research

Keywords: Penalized maximum likelihood; LASSO; SCAD; thresholding; post-model-selection estimator; finite-sample distribution; asymptotic distribution; estimation of distribution; uniform consistency;

Other versions of this item:

Find related papers by JEL classification:

This paper has been announced in the following NEP Reports:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Hannes Leeb & Benedikt M. Pötscher, 2003. "Performance Limits for Estimators of the Risk or Distribution of Shrinkage-Type Estimators, and Some General Lower Risk-Bound Results," Vienna Economics Papers 0301, University of Vienna, Department of Economics.
  2. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
  3. Knight, Keith, 2008. "Shrinkage Estimation For Nearly Singular Designs," Econometric Theory, Cambridge University Press, vol. 24(02), pages 323-337, April.
  4. Kabaila, Paul, 1995. "The Effect of Model Selection on Confidence Regions and Prediction Regions," Econometric Theory, Cambridge University Press, vol. 11(03), pages 537-549, June.
  5. Hannes Leeb & Benedikt M. Poetscher, 2000. "The Finite-Sample Distribution of Post-Model-Selection Estimators, and Uniform Versus Non-Uniform Approximations," Econometrics 0004001, EconWPA.
  6. Hannes Leeb & Benedikt M. Potscher, 2003. "Can One Estimate the Conditional Distribution of Post-Model-Selection Estimators?," Cowles Foundation Discussion Papers 1444, Cowles Foundation for Research in Economics, Yale University.
  7. Pötscher, Benedikt M., 2006. "The Distribution of Model Averaging Estimators and an Impossibility Result Regarding Its Estimation," MPRA Paper 73, University Library of Munich, Germany, revised Jul 2006.
  8. Hannes Leeb & Benedikt M. Poetscher, 2005. "Sparse Estimators and the Oracle Property, or the Return of Hodges' Estimator," Cowles Foundation Discussion Papers 1500, Cowles Foundation for Research in Economics, Yale University, revised Apr 2007.
  9. Pötscher, B.M., 1991. "Effects of Model Selection on Inference," Econometric Theory, Cambridge University Press, vol. 7(02), pages 163-185, June.
  10. Rudolf Beran, 1997. "Diagnosing Bootstrap Success," Annals of the Institute of Statistical Mathematics, Springer, vol. 49(1), pages 1-24, March.
  11. repec:cup:etheor:v:11:y:1995:i:3:p:537-49 is not listed on IDEAS
  12. Zou, Hui, 2006. "The Adaptive Lasso and Its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 1418-1429, December.
  13. Leeb, Hannes & P tscher, Benedikt M., 2005. "Model Selection And Inference: Facts And Fiction," Econometric Theory, Cambridge University Press, vol. 21(01), pages 21-59, February.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. Leeb, Hannes & Pötscher, Benedikt M. & Ewald, Karl, 2014. "On various confidence intervals post-model-selection," MPRA Paper 52858, University Library of Munich, Germany.
  2. Pötscher, Benedikt M. & Schneider, Ulrike, 2007. "On the distribution of the adaptive LASSO estimator," MPRA Paper 6913, University Library of Munich, Germany.
  3. Kascha, Christian & Trenkler, Carsten, 2011. "Bootstrapping the likelihood ratio cointegration test in error correction models with unknown lag order," Computational Statistics & Data Analysis, Elsevier, vol. 55(2), pages 1008-1017, February.
  4. Adam McCloskey, 2012. "Bonferroni-Based Size-Correction for Nonstandard Testing Problems," Working Papers 2012-16, Brown University, Department of Economics.
  5. Anders Bredahl Kock, 2012. "On the Oracle Property of the Adaptive Lasso in Stationary and Nonstationary Autoregressions," CREATES Research Papers 2012-05, School of Economics and Management, University of Aarhus.
  6. Anders Bredahl Kock, 2013. "Oracle inequalities for high-dimensional panel data models," CREATES Research Papers 2013-20, School of Economics and Management, University of Aarhus.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:5615. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Ekkehart Schlicht).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.