IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0148620.html
   My bibliography  Save this article

An Adaptive Ridge Procedure for L0 Regularization

Author

Listed:
  • Florian Frommlet
  • Grégory Nuel

Abstract

Penalized selection criteria like AIC or BIC are among the most popular methods for variable selection. Their theoretical properties have been studied intensively and are well understood, but making use of them in case of high-dimensional data is difficult due to the non-convex optimization problem induced by L0 penalties. In this paper we introduce an adaptive ridge procedure (AR), where iteratively weighted ridge problems are solved whose weights are updated in such a way that the procedure converges towards selection with L0 penalties. After introducing AR its specific shrinkage properties are studied in the particular case of orthogonal linear regression. Based on extensive simulations for the non-orthogonal case as well as for Poisson regression the performance of AR is studied and compared with SCAD and adaptive LASSO. Furthermore an efficient implementation of AR in the context of least-squares segmentation is presented. The paper ends with an illustrative example of applying AR to analyze GWAS data.

Suggested Citation

  • Florian Frommlet & Grégory Nuel, 2016. "An Adaptive Ridge Procedure for L0 Regularization," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-23, February.
    • Handle: RePEc:plo:pone00:0148620
    DOI: 10.1371/journal.pone.0148620
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0148620
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0148620&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0148620?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ziqi Chen & Man-Lai Tang & Wei Gao & Ning-Zhong Shi, 2014. "New Robust Variable Selection Methods for Linear Regression Models," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 41(3), pages 725-741, September.
    2. Karl W. Broman & Terence P. Speed, 2002. "A model selection approach for the identification of quantitative trait loci in experimental crosses," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(4), pages 641-656, October.
    3. Xiaoguang Wang & Junhui Fan, 2014. "Variable selection for multivariate generalized linear models," Journal of Applied Statistics, Taylor & Francis Journals, vol. 41(2), pages 393-406, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sascha A. Keweloh, 2023. "Uncertain Short-Run Restrictions and Statistically Identified Structural Vector Autoregressions," Papers 2303.13281, arXiv.org, revised Apr 2024.
    2. Ersin Yılmaz & Syed Ejaz Ahmed & Dursun Aydın, 2020. "A-Spline Regression for Fitting a Nonparametric Regression Function with Censored Data," Stats, MDPI, vol. 3(2), pages 1-17, May.
    3. Luke Smallman & William Underwood & Andreas Artemiou, 2020. "Simple Poisson PCA: an algorithm for (sparse) feature extraction with simultaneous dimension determination," Computational Statistics, Springer, vol. 35(2), pages 559-577, June.
    4. Zhao, Hui & Sun, Dayu & Li, Gang & Sun, Jianguo, 2019. "Simultaneous estimation and variable selection for incomplete event history studies," Journal of Multivariate Analysis, Elsevier, vol. 171(C), pages 350-361.
    5. Jian Huang & Yuling Jiao & Lican Kang & Jin Liu & Yanyan Liu & Xiliang Lu, 2022. "GSDAR: a fast Newton algorithm for $$\ell _0$$ ℓ 0 regularized generalized linear models with statistical guarantee," Computational Statistics, Springer, vol. 37(1), pages 507-533, March.
    6. Goepp, Vivien & van de Kassteele, Jan, 2024. "Graph-based spatial segmentation of areal data," Computational Statistics & Data Analysis, Elsevier, vol. 192(C).
    7. Ikram Allam & Delphine Flatters & Géraldine Caumes & Leslie Regad & Vincent Delos & Gregory Nuel & Anne-Claude Camproux, 2018. "SAFlex: A structural alphabet extension to integrate protein structural flexibility and missing data information," PLOS ONE, Public Library of Science, vol. 13(7), pages 1-22, July.
    8. Jeongjin Lee & Taehwa Choi & Sangbum Choi, 2024. "Censored broken adaptive ridge regression in high-dimension," Computational Statistics, Springer, vol. 39(6), pages 3457-3482, September.
    9. Takumi Saegusa & Tianzhou Ma & Gang Li & Ying Qing Chen & Mei-Ling Ting Lee, 2020. "Variable Selection in Threshold Regression Model with Applications to HIV Drug Adherence Data," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 12(3), pages 376-398, December.
    10. Goepp, Vivien & Bouaziz, Olivier & Nuel, Grégory, 2025. "Spline regression with automatic knot selection," Computational Statistics & Data Analysis, Elsevier, vol. 202(C).
    11. Antonis Christou & Andreas Artemiou, 2023. "Adaptive L0 Regularization for Sparse Support Vector Regression," Mathematics, MDPI, vol. 11(13), pages 1-12, June.
    12. Philippe Goulet Coulombe, 2020. "Time-Varying Parameters as Ridge Regressions," Papers 2009.00401, arXiv.org, revised Nov 2024.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Frommlet, Florian & Ruhaltinger, Felix & Twaróg, Piotr & Bogdan, Małgorzata, 2012. "Modified versions of Bayesian Information Criterion for genome-wide association studies," Computational Statistics & Data Analysis, Elsevier, vol. 56(5), pages 1038-1051.
    2. Zak-Szatkowska, Malgorzata & Bogdan, Malgorzata, 2011. "Modified versions of the Bayesian Information Criterion for sparse Generalized Linear Models," Computational Statistics & Data Analysis, Elsevier, vol. 55(11), pages 2908-2924, November.
    3. Umberto Amato & Anestis Antoniadis & Italia De Feis & Irene Gijbels, 2021. "Penalised robust estimators for sparse and high-dimensional linear models," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(1), pages 1-48, March.
    4. Marit Ackermann & Mathieu Clément-Ziza & Jacob J Michaelson & Andreas Beyer, 2012. "Teamwork: Improved eQTL Mapping Using Combinations of Machine Learning Methods," PLOS ONE, Public Library of Science, vol. 7(7), pages 1-8, July.
    5. Erich Dolejsi & Bernhard Bodenstorfer & Florian Frommlet, 2014. "Analyzing Genome-Wide Association Studies with an FDR Controlling Modification of the Bayesian Information Criterion," PLOS ONE, Public Library of Science, vol. 9(7), pages 1-13, July.
    6. C. M. Kendziorski & M. Chen & M. Yuan & H. Lan & A. D. Attie, 2006. "Statistical Methods for Expression Quantitative Trait Loci (eQTL) Mapping," Biometrics, The International Biometric Society, vol. 62(1), pages 19-27, March.
    7. Yawei He & Zehua Chen, 2016. "The EBIC and a sequential procedure for feature selection in interactive linear models with high-dimensional data," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 68(1), pages 155-180, February.
    8. Chun Wang, 2021. "Using Penalized EM Algorithm to Infer Learning Trajectories in Latent Transition CDM," Psychometrika, Springer;The Psychometric Society, vol. 86(1), pages 167-189, March.
    9. Małgorzata Bogdan & Florian Frommlet & Przemysław Biecek & Riyan Cheng & Jayanta K. Ghosh & R.W. Doerge, 2008. "Extending the Modified Bayesian Information Criterion (mBIC) to Dense Markers and Multiple Interval Mapping," Biometrics, The International Biometric Society, vol. 64(4), pages 1162-1169, December.
    10. Baierl, Andreas & Futschik, Andreas & Bogdan, Malgorzata & Biecek, Przemyslaw, 2007. "Locating multiple interacting quantitative trait loci using robust model selection," Computational Statistics & Data Analysis, Elsevier, vol. 51(12), pages 6423-6434, August.
    11. Jiming Jiang & Thuan Nguyen & J. Sunil Rao, 2015. "The E-MS Algorithm: Model Selection With Incomplete Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(511), pages 1136-1147, September.
    12. Sun, Hanmei & Nguyen, Thuan & Luan, Yihui & Jiang, Jiming, 2018. "Classified mixed logistic model prediction," Journal of Multivariate Analysis, Elsevier, vol. 168(C), pages 63-74.
    13. Ziqi Chen & Jing Ning & Yu Shen & Jing Qin, 2021. "Combining primary cohort data with external aggregate information without assuming comparability," Biometrics, The International Biometric Society, vol. 77(3), pages 1024-1036, September.
    14. Wang, Tao & Zhu, Lixing, 2011. "Consistent tuning parameter selection in high dimensional sparse linear regression," Journal of Multivariate Analysis, Elsevier, vol. 102(7), pages 1141-1151, August.
    15. Shriner, Daniel & Yi, Nengjun, 2009. "Deviance information criterion (DIC) in Bayesian multiple QTL mapping," Computational Statistics & Data Analysis, Elsevier, vol. 53(5), pages 1850-1860, March.
    16. Edward Boone & Susan Simmons & Haikun Bao & Ann Stapleton, 2008. "Bayesian hierarchical regression models for detecting QTLs in plant experiments," Journal of Applied Statistics, Taylor & Francis Journals, vol. 35(7), pages 799-808.
    17. Eduardo F. Mendes & Gabriel J. P. Pinto, 2023. "Generalized Information Criteria for Structured Sparse Models," Papers 2309.01764, arXiv.org.
    18. Chen, Yang & Luo, Ziyan & Kong, Lingchen, 2021. "ℓ2,0-norm based selection and estimation for multivariate generalized linear models," Journal of Multivariate Analysis, Elsevier, vol. 185(C).
    19. Hanwen Huang & Haibo Zhou & Fuxia Cheng & Ina Hoeschele & Fei Zou, 2010. "Gaussian Process Based Bayesian Semiparametric Quantitative Trait Loci Interval Mapping," Biometrics, The International Biometric Society, vol. 66(1), pages 222-232, March.
    20. Frommlet Florian & Ljubic Ivana & Arnardóttir Helga Björk & Bogdan Malgorzata, 2012. "QTL Mapping Using a Memetic Algorithm with Modifications of BIC as Fitness Function," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 11(4), pages 1-26, May.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0148620. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.