IDEAS home Printed from https://ideas.repec.org/a/eee/stapro/v81y2011i12p1760-1765.html
   My bibliography  Save this article

Generalization of ℓ1 constraints for high dimensional regression problems

Author

Listed:
  • Alquier, Pierre
  • Hebiri, Mohamed

Abstract

We focus on the high dimensional linear regression Y∼N(Xβ∗,σ2In), where β∗∈Rp is the parameter of interest. In this setting, several estimators such as the LASSO (Tibshirani, 1996) and the Dantzig Selector (Candes and Tao, 2007) are known to satisfy interesting properties whenever the vector β∗ is sparse. Interestingly, both the LASSO and the Dantzig Selector can be seen as orthogonal projections of 0 into DC(s)={β∈Rp,‖X′(Y−Xβ)‖∞≤s}, using an ℓ1 distance for the Dantzig Selector and ℓ2 for the LASSO. For a well chosen s>0, this set is actually a confidence region for β∗. In this paper, we investigate the properties of estimators defined as projections on DC(s) using general distances. We prove that the obtained estimators satisfy oracle properties close to the one of the LASSO and the Dantzig Selector. On top of that, it turns out that these estimators can be tuned to exploit a different sparsity or/and slightly different estimation objectives.

Suggested Citation

  • Alquier, Pierre & Hebiri, Mohamed, 2011. "Generalization of ℓ1 constraints for high dimensional regression problems," Statistics & Probability Letters, Elsevier, vol. 81(12), pages 1760-1765.
  • Handle: RePEc:eee:stapro:v:81:y:2011:i:12:p:1760-1765
    DOI: 10.1016/j.spl.2011.07.011
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167715211002422
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.spl.2011.07.011?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Alexandre Belloni & Victor Chernozhukov, 2011. "High Dimensional Sparse Econometric Models: An Introduction," Papers 1106.5242, arXiv.org, revised Sep 2011.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Victor Chernozhukov & Denis Chetverikov & Kengo Kato, 2012. "Central limit theorems and multiplier bootstrap when p is much larger than n," CeMMAP working papers 45/12, Institute for Fiscal Studies.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ning Xu & Jian Hong & Timothy C. G. Fisher, 2016. "Model selection consistency from the perspective of generalization ability and VC theory with an application to Lasso," Papers 1606.00142, arXiv.org.
    2. Yamada, Hiroshi & Yoon, Gawon, 2014. "When Grilli and Yang meet Prebisch and Singer: Piecewise linear trends in primary commodity prices," Journal of International Money and Finance, Elsevier, vol. 42(C), pages 193-207.
    3. Damian Kozbur, 2013. "Inference in additively separable models with a high-dimensional set of conditioning variables," ECON - Working Papers 284, Department of Economics - University of Zurich, revised Apr 2018.
    4. Myrto Kalouptsidi, 2014. "Detection and Impact of Industrial Subsidies: The Case of World Shipbuilding," NBER Working Papers 20119, National Bureau of Economic Research, Inc.
    5. de Paula, Aureo & Rasul, Imran & Souza, Pedro, 2018. "Identifying Network Ties from Panel Data: Theory and an Application to Tax Competition," CEPR Discussion Papers 12792, C.E.P.R. Discussion Papers.
    6. Alexandre Belloni & Victor Chernozhukov & Lie Wang, 2013. "Pivotal estimation via square-root lasso in nonparametric regression," CeMMAP working papers CWP62/13, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    7. Malene Kallestrup-Lamb & Anders Bredahl Kock & Johannes Tang Kristensen, 2016. "Lassoing the Determinants of Retirement," Econometric Reviews, Taylor & Francis Journals, vol. 35(8-10), pages 1522-1561, December.
    8. Achim Ahrens & Christian B. Hansen & Mark E. Schaffer, 2020. "lassopack: Model selection and prediction with regularized regression in Stata," Stata Journal, StataCorp LP, vol. 20(1), pages 176-235, March.
    9. Ulrike Schneider, 2016. "Confidence Sets Based on Thresholding Estimators in High-Dimensional Gaussian Regression Models," Econometric Reviews, Taylor & Francis Journals, vol. 35(8-10), pages 1412-1455, December.
    10. Achim Ahrens & Arnab Bhattacharjee, 2015. "Two-Step Lasso Estimation of the Spatial Weights Matrix," Econometrics, MDPI, vol. 3(1), pages 1-28, March.
    11. Denis Chetverikov & . ., 2016. "On cross-validated Lasso," CeMMAP working papers CWP47/16, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    12. Kock, Anders Bredahl, 2016. "Oracle inequalities, variable selection and uniform inference in high-dimensional correlated random effects panel data models," Journal of Econometrics, Elsevier, vol. 195(1), pages 71-85.
    13. Federico A. Bugni & Mehmet Caner & Anders Bredahl Kock & Soumendra Lahiri, 2016. "Inference in partially identified models with many moment inequalities using Lasso," CREATES Research Papers 2016-12, Department of Economics and Business Economics, Aarhus University.
    14. Daniel Felix Ahelegbey & Monica Billio & Roberto Casarin, 2016. "Sparse Graphical Vector Autoregression: A Bayesian Approach," Annals of Economics and Statistics, GENES, issue 123-124, pages 333-361.
    15. Laurent Callot & Johannes Tang Kristensen, 2014. "Vector Autoregressions with Parsimoniously Time Varying Parameters and an Application to Monetary Policy," CREATES Research Papers 2014-41, Department of Economics and Business Economics, Aarhus University.
    16. Simon Freyaldenhoven, 2020. "Identification Through Sparsity in Factor Models," Working Papers 20-25, Federal Reserve Bank of Philadelphia.
    17. André Nunes Maranhão & Nicole Rennó Castro, 2023. "Dissecting Brazilian agriculture business cycles in high-dimensional and time-irregular span contexts," Empirical Economics, Springer, vol. 65(4), pages 1543-1578, October.
    18. Newhouse,David Locke & Merfeld,Joshua David & Ramakrishnan,Anusha Pudugramam & Swartz,Tom & Lahiri,Partha, 2022. "Small Area Estimation of Monetary Poverty in Mexico Using Satellite Imagery and Machine Learning," Policy Research Working Paper Series 10175, The World Bank.
    19. Masaki,Takaaki & Newhouse,David Locke & Silwal,Ani Rudra & Bedada,Adane & Engstrom,Ryan, 2020. "Small Area Estimation of Non-Monetary Poverty with Geospatial Data," Policy Research Working Paper Series 9383, The World Bank.
    20. Ng, Serena, 2013. "Variable Selection in Predictive Regressions," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 752-789, Elsevier.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:stapro:v:81:y:2011:i:12:p:1760-1765. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/622892/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.