IDEAS home Printed from https://ideas.repec.org/a/spr/stpapr/v66y2025i4d10.1007_s00362-025-01700-2.html
   My bibliography  Save this article

From screening to variable selection by an iterative nonparametric procedure based on derivatives

Author

Listed:
  • Francesco Giordano

    (University of Salerno)

  • Sara Milito

    (University of Salerno)

  • Maria Lucia Parrella

    (University of Salerno)

Abstract

To identify the true relevant predictors in a nonparametric regression model with ultra-high-dimensional data, we propose a general iterative procedure that is able to transform a screening procedure into a variable selection one, without requiring the number of true relevant variables to be finite. This fully nonparametric procedure is based on a novel combination of DELSIS (a model-free variable screening method based on empirical likelihood and derivative estimation) and PenGAM (a penalised variable selection method based on splines). The main advantage of the new proposal is its robustness to the presence of correlation among the predictors and its capability to correctly identify the whole set of relevant covariates, including those marginally uncorrelated but jointly related to the response (“hidden covariates”) and those with low signal (“weak covariates”), much better than the main alternative approaches. From a theoretical point of view, we show the consistency of the proposal and its faster estimation rate with respect to the classical penalised approaches. Finally, we illustrate the performance of the new procedure through some simulations and an empirical analysis.

Suggested Citation

  • Francesco Giordano & Sara Milito & Maria Lucia Parrella, 2025. "From screening to variable selection by an iterative nonparametric procedure based on derivatives," Statistical Papers, Springer, vol. 66(4), pages 1-35, June.
    • Handle: RePEc:spr:stpapr:v:66:y:2025:i:4:d:10.1007_s00362-025-01700-2
    DOI: 10.1007/s00362-025-01700-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00362-025-01700-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00362-025-01700-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Fan, Jianqing & Feng, Yang & Song, Rui, 2011. "Nonparametric Independence Screening in Sparse Ultra-High-Dimensional Additive Models," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 544-557.
    2. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    3. Jianqing Fan & Jinchi Lv, 2008. "Sure independence screening for ultrahigh dimensional feature space," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(5), pages 849-911, November.
    4. F. Giordano & S. Milito & M. L. Parrella, 2022. "A nonparametric procedure for linear and nonlinear variable screening," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 34(4), pages 859-894, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shan Luo & Zehua Chen, 2014. "Sequential Lasso Cum EBIC for Feature Selection With Ultra-High Dimensional Feature Space," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(507), pages 1229-1240, September.
    2. Wang, Christina Dan & Chen, Zhao & Lian, Yimin & Chen, Min, 2022. "Asset selection based on high frequency Sharpe ratio," Journal of Econometrics, Elsevier, vol. 227(1), pages 168-188.
    3. Loann David Denis Desboulets, 2018. "A Review on Variable Selection in Regression Analysis," Econometrics, MDPI, vol. 6(4), pages 1-27, November.
    4. Li, Xinyi & Wang, Li & Nettleton, Dan, 2019. "Sparse model identification and learning for ultra-high-dimensional additive partially linear models," Journal of Multivariate Analysis, Elsevier, vol. 173(C), pages 204-228.
    5. Jingyuan Liu & Runze Li & Rongling Wu, 2014. "Feature Selection for Varying Coefficient Models With Ultrahigh-Dimensional Covariates," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(505), pages 266-274, March.
    6. Jianqing Fan & Yang Feng & Jiancheng Jiang & Xin Tong, 2016. "Feature Augmentation via Nonparametrics and Selection (FANS) in High-Dimensional Classification," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(513), pages 275-287, March.
    7. Jingxuan Luo & Lili Yue & Gaorong Li, 2023. "Overview of High-Dimensional Measurement Error Regression Models," Mathematics, MDPI, vol. 11(14), pages 1-22, July.
    8. Li, Degui & Linton, Oliver & Lu, Zudi, 2015. "A flexible semiparametric forecasting model for time series," Journal of Econometrics, Elsevier, vol. 187(1), pages 345-357.
    9. Xiaochao Xia & Hao Ming, 2022. "A Flexibly Conditional Screening Approach via a Nonparametric Quantile Partial Correlation," Mathematics, MDPI, vol. 10(24), pages 1-32, December.
    10. Zhao, Bangxin & Liu, Xin & He, Wenqing & Yi, Grace Y., 2021. "Dynamic tilted current correlation for high dimensional variable screening," Journal of Multivariate Analysis, Elsevier, vol. 182(C).
    11. Zhenghui Feng & Lu Lin & Ruoqing Zhu & Lixing Zhu, 2020. "Nonparametric variable selection and its application to additive models," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 72(3), pages 827-854, June.
    12. He, Yong & Zhang, Liang & Ji, Jiadong & Zhang, Xinsheng, 2019. "Robust feature screening for elliptical copula regression model," Journal of Multivariate Analysis, Elsevier, vol. 173(C), pages 568-582.
    13. Congran Yu & Wenwen Guo & Xinyuan Song & Hengjian Cui, 2023. "Feature screening with latent responses," Biometrics, The International Biometric Society, vol. 79(2), pages 878-890, June.
    14. Min Chen & Yimin Lian & Zhao Chen & Zhengjun Zhang, 2017. "Sure explained variability and independence screening," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 29(4), pages 849-883, October.
    15. Zhang, Shucong & Zhou, Yong, 2018. "Variable screening for ultrahigh dimensional heterogeneous data via conditional quantile correlations," Journal of Multivariate Analysis, Elsevier, vol. 165(C), pages 1-13.
    16. Haofeng Wang & Hongxia Jin & Xuejun Jiang & Jingzhi Li, 2022. "Model Selection for High Dimensional Nonparametric Additive Models via Ridge Estimation," Mathematics, MDPI, vol. 10(23), pages 1-22, December.
    17. Cui, Wenquan & Cheng, Haoyang & Sun, Jiajing, 2018. "An RKHS-based approach to double-penalized regression in high-dimensional partially linear models," Journal of Multivariate Analysis, Elsevier, vol. 168(C), pages 201-210.
    18. Zhang, Shucong & Pan, Jing & Zhou, Yong, 2018. "Robust conditional nonparametric independence screening for ultrahigh-dimensional data," Statistics & Probability Letters, Elsevier, vol. 143(C), pages 95-101.
    19. Peirong Xu & Lixing Zhu & Yi Li, 2014. "Ultrahigh dimensional time course feature selection," Biometrics, The International Biometric Society, vol. 70(2), pages 356-365, June.
    20. Yang, Baoying & Yin, Xiangrong & Zhang, Nan, 2019. "Sufficient variable selection using independence measures for continuous response," Journal of Multivariate Analysis, Elsevier, vol. 173(C), pages 480-493.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stpapr:v:66:y:2025:i:4:d:10.1007_s00362-025-01700-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.