IDEAS home Printed from https://ideas.repec.org/a/eee/jmvana/v100y2009i9p2100-2111.html
   My bibliography  Save this article

Automatic model selection for partially linear models

Author

Listed:
  • Ni, Xiao
  • Zhang, Hao Helen
  • Zhang, Daowen

Abstract

We propose and study a unified procedure for variable selection in partially linear models. A new type of double-penalized least squares is formulated, using the smoothing spline to estimate the nonparametric part and applying a shrinkage penalty on parametric components to achieve model parsimony. Theoretically we show that, with proper choices of the smoothing and regularization parameters, the proposed procedure can be as efficient as the oracle estimator [J. Fan, R. Li, Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of American Statistical Association 96 (2001) 1348-1360]. We also study the asymptotic properties of the estimator when the number of parametric effects diverges with the sample size. Frequentist and Bayesian estimates of the covariance and confidence intervals are derived for the estimators. One great advantage of this procedure is its linear mixed model (LMM) representation, which greatly facilitates its implementation by using standard statistical software. Furthermore, the LMM framework enables one to treat the smoothing parameter as a variance component and hence conveniently estimate it together with other regression coefficients. Extensive numerical studies are conducted to demonstrate the effective performance of the proposed procedure.

Suggested Citation

  • Ni, Xiao & Zhang, Hao Helen & Zhang, Daowen, 2009. "Automatic model selection for partially linear models," Journal of Multivariate Analysis, Elsevier, vol. 100(9), pages 2100-2111, October.
    • Handle: RePEc:eee:jmvana:v:100:y:2009:i:9:p:2100-2111
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0047-259X(09)00117-1
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Zou, Hui, 2006. "The Adaptive Lasso and Its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 1418-1429, December.
    2. Liang, Hua, 2006. "Estimation in partially linear models and numerical comparisons," Computational Statistics & Data Analysis, Elsevier, vol. 50(3), pages 675-687, February.
    3. Jianqing Fan & Runze Li, 2004. "New Estimation and Model Selection Procedures for Semiparametric Modeling in Longitudinal Data Analysis," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 710-723, January.
    4. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    5. Ruppert,David & Wand,M. P. & Carroll,R. J., 2003. "Semiparametric Regression," Cambridge Books, Cambridge University Press, number 9780521785167.
    6. Ruppert,David & Wand,M. P. & Carroll,R. J., 2003. "Semiparametric Regression," Cambridge Books, Cambridge University Press, number 9780521780506.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Du, Pang & Cheng, Guang & Liang, Hua, 2012. "Semiparametric regression models with additive nonparametric components and high dimensional parametric components," Computational Statistics & Data Analysis, Elsevier, vol. 56(6), pages 2006-2017.
    2. Cui, Wenquan & Cheng, Haoyang & Sun, Jiajing, 2018. "An RKHS-based approach to double-penalized regression in high-dimensional partially linear models," Journal of Multivariate Analysis, Elsevier, vol. 168(C), pages 201-210.
    3. Guozhi Hu & Weihu Cheng & Jie Zeng, 2023. "Optimal Model Averaging for Semiparametric Partially Linear Models with Censored Data," Mathematics, MDPI, vol. 11(3), pages 1-21, February.
    4. Mingqiu Wang & Guo-Liang Tian, 2016. "Robust group non-convex estimations for high-dimensional partially linear models," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 28(1), pages 49-67, March.
    5. Loann David Denis Desboulets, 2018. "A Review on Variable Selection in Regression Analysis," Econometrics, MDPI, vol. 6(4), pages 1-27, November.
    6. Aifen Feng & Xiaogai Chang & Youlin Shang & Jingya Fan, 2022. "Application of the ADMM Algorithm for a High-Dimensional Partially Linear Model," Mathematics, MDPI, vol. 10(24), pages 1-13, December.
    7. Arfan Raheen Afzal & Jing Yang & Xuewen Lu, 2021. "Variable selection in partially linear additive hazards model with grouped covariates and a diverging number of parameters," Computational Statistics, Springer, vol. 36(2), pages 829-855, June.
    8. Germán Aneiros & Philippe Vieu, 2015. "Partial linear modelling with multi-functional covariates," Computational Statistics, Springer, vol. 30(3), pages 647-671, September.
    9. Guang Cheng & Hao Zhang & Zuofeng Shang, 2015. "Sparse and efficient estimation for partial spline models with increasing dimension," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 67(1), pages 93-127, February.
    10. Hu Yang & Ning Li & Jing Yang, 2020. "A robust and efficient estimation and variable selection method for partially linear models with large-dimensional covariates," Statistical Papers, Springer, vol. 61(5), pages 1911-1937, October.
    11. Feng Li & Lu Lin & Yuxia Su, 2013. "Variable selection and parameter estimation for partially linear models via Dantzig selector," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 76(2), pages 225-238, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guang Cheng & Hao Zhang & Zuofeng Shang, 2015. "Sparse and efficient estimation for partial spline models with increasing dimension," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 67(1), pages 93-127, February.
    2. Yazhao Lv & Riquan Zhang & Weihua Zhao & Jicai Liu, 2015. "Quantile regression and variable selection of partial linear single-index model," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 67(2), pages 375-409, April.
    3. Feng Li & Lu Lin & Yuxia Su, 2013. "Variable selection and parameter estimation for partially linear models via Dantzig selector," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 76(2), pages 225-238, February.
    4. Xiao Ni & Daowen Zhang & Hao Helen Zhang, 2010. "Variable Selection for Semiparametric Mixed Models in Longitudinal Studies," Biometrics, The International Biometric Society, vol. 66(1), pages 79-88, March.
    5. Liu, Jingyuan & Lou, Lejia & Li, Runze, 2018. "Variable selection for partially linear models via partial correlation," Journal of Multivariate Analysis, Elsevier, vol. 167(C), pages 418-434.
    6. Gerhard Tutz & Margret-Ruth Oelker, 2017. "Modelling Clustered Heterogeneity: Fixed Effects, Random Effects and Mixtures," International Statistical Review, International Statistical Institute, vol. 85(2), pages 204-227, August.
    7. Degao Li & Guodong Li & Jinhong You, 2014. "Significant Variable Selection And Autoregressive Order Determination For Time-Series Partially Linear Models," Journal of Time Series Analysis, Wiley Blackwell, vol. 35(5), pages 478-490, August.
    8. Peng, Heng & Lu, Ying, 2012. "Model selection in linear mixed effect models," Journal of Multivariate Analysis, Elsevier, vol. 109(C), pages 109-129.
    9. Shan Luo & Zehua Chen, 2014. "Sequential Lasso Cum EBIC for Feature Selection With Ultra-High Dimensional Feature Space," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(507), pages 1229-1240, September.
    10. Lai, Peng & Wang, Qihua & Lian, Heng, 2012. "Bias-corrected GEE estimation and smooth-threshold GEE variable selection for single-index models with clustered data," Journal of Multivariate Analysis, Elsevier, vol. 105(1), pages 422-432.
    11. Joseph G. Ibrahim & Hongtu Zhu & Ramon I. Garcia & Ruixin Guo, 2011. "Fixed and Random Effects Selection in Mixed Effects Models," Biometrics, The International Biometric Society, vol. 67(2), pages 495-503, June.
    12. Jingyuan Liu & Runze Li & Rongling Wu, 2014. "Feature Selection for Varying Coefficient Models With Ultrahigh-Dimensional Covariates," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(505), pages 266-274, March.
    13. Zhangong Zhou & Rong Jiang & Weimin Qian, 2013. "LAD variable selection for linear models with randomly censored data," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 76(2), pages 287-300, February.
    14. Runze Li & Lei Nie, 2008. "Efficient Statistical Inference Procedures for Partially Nonlinear Models and their Applications," Biometrics, The International Biometric Society, vol. 64(3), pages 904-911, September.
    15. Xingwei Tong & Xin He & Liuquan Sun & Jianguo Sun, 2009. "Variable Selection for Panel Count Data via Non‐Concave Penalized Estimating Function," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 36(4), pages 620-635, December.
    16. Cui, Wenquan & Cheng, Haoyang & Sun, Jiajing, 2018. "An RKHS-based approach to double-penalized regression in high-dimensional partially linear models," Journal of Multivariate Analysis, Elsevier, vol. 168(C), pages 201-210.
    17. Takuma Yoshida, 2019. "Two stage smoothing in additive models with missing covariates," Statistical Papers, Springer, vol. 60(6), pages 1803-1826, December.
    18. Peixin Zhao & Liugen Xue, 2012. "Variable selection in semiparametric regression analysis for longitudinal data," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 64(1), pages 213-231, February.
    19. Leeb, Hannes & Potscher, Benedikt M., 2008. "Sparse estimators and the oracle property, or the return of Hodges' estimator," Journal of Econometrics, Elsevier, vol. 142(1), pages 201-211, January.
    20. Gerhard Tutz & Jan Gertheiss, 2014. "Rating Scales as Predictors—The Old Question of Scale Level and Some Answers," Psychometrika, Springer;The Psychometric Society, vol. 79(3), pages 357-376, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jmvana:v:100:y:2009:i:9:p:2100-2111. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/622892/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.