IDEAS home Printed from https://ideas.repec.org/a/spr/stpapr/v62y2021i2d10.1007_s00362-019-01107-w.html
   My bibliography  Save this article

Nonnegative estimation and variable selection under minimax concave penalty for sparse high-dimensional linear regression models

Author

Listed:
  • Ning Li

    (Chongqing University)

  • Hu Yang

    (Chongqing University)

Abstract

As a promising alternative to the Lasso penalty, the minimax concave penalty (MCP) produces nearly unbiased shrinkage estimate. In this paper, we propose the nonnegative MCP estimator in the specific high-dimensional settings where the regression coefficients are imposed with the nonnegativity constraint, which is particularly relevant when modelling nonnegative data. We prove that the asymptotic theory for the nonnegative MCP estimator requires much weaker assumptions than those in the literature. In particular, we do not impose the restrictive distributional conditions on the random errors and do not assume the irrepresentable condition or its variants on the design matrix. For the nonnegative MCP estimator, despite its good asymptotic properties, the corresponding optimization problem is non-convex, and consequently much hard to solve. Thus, we develop an efficient computation algorithm, which is a coupling of the difference convex algorithm with the multiplicative updates algorithm, for the implementation of the nonnegative MCP estimator. Simulation studies are carried out to examine superior performance of the nonnegative MCP estimator against alternative methods and a real data analysis for index tracking is also presented for illustration.

Suggested Citation

  • Ning Li & Hu Yang, 2021. "Nonnegative estimation and variable selection under minimax concave penalty for sparse high-dimensional linear regression models," Statistical Papers, Springer, vol. 62(2), pages 661-680, April.
    • Handle: RePEc:spr:stpapr:v:62:y:2021:i:2:d:10.1007_s00362-019-01107-w
    DOI: 10.1007/s00362-019-01107-w
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00362-019-01107-w
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00362-019-01107-w?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    2. Ming Yuan & Yi Lin, 2007. "On the non‐negative garrotte estimator," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(2), pages 143-161, April.
    3. Wu, Lan & Yang, Yuehan & Liu, Hanzhong, 2014. "Nonnegative-lasso and application in index tracking," Computational Statistics & Data Analysis, Elsevier, vol. 70(C), pages 116-126.
    4. N. Neykov & P. Filzmoser & P. Neytchev, 2014. "Erratum to: Ultrahigh dimensional variable selection through the penalized maximum trimmed likelihood estimator," Statistical Papers, Springer, vol. 55(3), pages 917-918, August.
    5. YichaoWu, & Liu, Yufeng, 2007. "Robust Truncated Hinge Loss Support Vector Machines," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 974-983, September.
    6. Hu Yang & Jianwen Xu, 2009. "An alternative stochastic restricted Liu estimator in linear regression," Statistical Papers, Springer, vol. 50(3), pages 639-647, June.
    7. Yalian Li & Hu Yang, 2012. "A new Liu-type estimator in linear regression model," Statistical Papers, Springer, vol. 53(2), pages 427-437, May.
    8. A. Antoniadis & I. Gijbels & S. Lambert-Lacroix, 2014. "Penalized estimation in additive varying coefficient models using grouped regularization," Statistical Papers, Springer, vol. 55(3), pages 727-750, August.
    9. Kim, Yongdai & Choi, Hosik & Oh, Hee-Seok, 2008. "Smoothly Clipped Absolute Deviation on High Dimensions," Journal of the American Statistical Association, American Statistical Association, vol. 103(484), pages 1665-1673.
    10. N. Neykov & P. Filzmoser & P. Neytchev, 2014. "Ultrahigh dimensional variable selection through the penalized maximum trimmed likelihood estimator," Statistical Papers, Springer, vol. 55(1), pages 187-207, February.
    11. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    12. N. J. Jobst & M. D. Horniman & C. A. Lucas & G. Mitra, 2001. "Computational aspects of alternative portfolio selection models in the presence of discrete asset choice constraints," Quantitative Finance, Taylor & Francis Journals, vol. 1(5), pages 489-501.
    13. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xia, Siwei & Yang, Yuehan & Yang, Hu, 2023. "High-dimensional sparse portfolio selection with nonnegative constraint," Applied Mathematics and Computation, Elsevier, vol. 443(C).
    2. Jan Pablo Burgard & Joscha Krause & Dennis Kreber & Domingo Morales, 2021. "The generalized equivalence of regularization and min–max robustification in linear mixed models," Statistical Papers, Springer, vol. 62(6), pages 2857-2883, December.
    3. Loann David Denis Desboulets, 2018. "A Review on Variable Selection in Regression Analysis," Econometrics, MDPI, vol. 6(4), pages 1-27, November.
    4. Ricardo P. Masini & Marcelo C. Medeiros & Eduardo F. Mendes, 2023. "Machine learning advances for time series forecasting," Journal of Economic Surveys, Wiley Blackwell, vol. 37(1), pages 76-111, February.
    5. Mogliani, Matteo & Simoni, Anna, 2021. "Bayesian MIDAS penalized regressions: Estimation, selection, and prediction," Journal of Econometrics, Elsevier, vol. 222(1), pages 833-860.
    6. Caiya Zhang & Yanbiao Xiang, 2016. "On the oracle property of adaptive group Lasso in high-dimensional linear models," Statistical Papers, Springer, vol. 57(1), pages 249-265, March.
    7. Xiaofei Wu & Rongmei Liang & Hu Yang, 2022. "Penalized and constrained LAD estimation in fixed and high dimension," Statistical Papers, Springer, vol. 63(1), pages 53-95, February.
    8. Murat Genç, 2022. "A new double-regularized regression using Liu and lasso regularization," Computational Statistics, Springer, vol. 37(1), pages 159-227, March.
    9. Fang, Xiaolei & Paynabar, Kamran & Gebraeel, Nagi, 2017. "Multistream sensor fusion-based prognostics model for systems with single failure modes," Reliability Engineering and System Safety, Elsevier, vol. 159(C), pages 322-331.
    10. Siwei Xia & Yuehan Yang & Hu Yang, 2022. "Sparse Laplacian Shrinkage with the Graphical Lasso Estimator for Regression Problems," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 31(1), pages 255-277, March.
    11. Kwon, Sunghoon & Oh, Seungyoung & Lee, Youngjo, 2016. "The use of random-effect models for high-dimensional variable selection problems," Computational Statistics & Data Analysis, Elsevier, vol. 103(C), pages 401-412.
    12. Belli, Edoardo, 2022. "Smoothly adaptively centered ridge estimator," Journal of Multivariate Analysis, Elsevier, vol. 189(C).
    13. Alena Skolkova, 2023. "Instrumental Variable Estimation with Many Instruments Using Elastic-Net IV," CERGE-EI Working Papers wp759, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
    14. Mike K. P. So & Wing Ki Liu & Amanda M. Y. Chu, 2018. "Bayesian Shrinkage Estimation Of Time-Varying Covariance Matrices In Financial Time Series," Advances in Decision Sciences, Asia University, Taiwan, vol. 22(1), pages 369-404, December.
    15. Gerda Claeskens, 2012. "Focused estimation and model averaging with penalization methods: an overview," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 66(3), pages 272-287, August.
    16. Wang, Jia & Cai, Xizhen & Li, Runze, 2021. "Variable selection for partially linear models via Bayesian subset modeling with diffusing prior," Journal of Multivariate Analysis, Elsevier, vol. 183(C).
    17. Wei Sun & Lexin Li, 2012. "Multiple Loci Mapping via Model-free Variable Selection," Biometrics, The International Biometric Society, vol. 68(1), pages 12-22, March.
    18. Mingqiu Wang & Guo-Liang Tian, 2016. "Robust group non-convex estimations for high-dimensional partially linear models," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 28(1), pages 49-67, March.
    19. Yanxin Wang & Qibin Fan & Li Zhu, 2018. "Variable selection and estimation using a continuous approximation to the $$L_0$$ L 0 penalty," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 70(1), pages 191-214, February.
    20. Li Liu & Hao Wang & Yanyan Liu & Jian Huang, 2021. "Model pursuit and variable selection in the additive accelerated failure time model," Statistical Papers, Springer, vol. 62(6), pages 2627-2659, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stpapr:v:62:y:2021:i:2:d:10.1007_s00362-019-01107-w. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.