IDEAS home Printed from https://ideas.repec.org/a/spr/stmapp/v24y2015i1p41-60.html
   My bibliography  Save this article

A group VISA algorithm for variable selection

Author

Listed:
  • Abdallah Mkhadri

    ()

  • Mohamed Ouhourane

    ()

Abstract

We consider the problem of selecting grouped variables in a linear regression model based on penalized least squares. The group-Lasso and the group-Lars procedures are designed for automatically performing both the shrinkage and the selection of important groups of variables. However, since the same tuning parameter is used (as in Lasso or Lars ) for both group variable selection and shrinkage coefficients, it can lead to over shrinkage the significant groups of variables or inclusion of many irrelevant groups of predictors. This situation occurs when the true number of non-zero groups of coefficients is small relative to the number $$p$$ p of variables. We introduce a novel sparse regression method, called the Group-VISA (GVISA), which extends the VISA effect to grouped variables. It combines the idea of VISA algorithm which avoids the over shrinkage problem of regression coefficients and the idea of the GLars-type estimator which shrinks and selects the members of the group together. Hence, GVISA is able to select a sparse group model by avoiding the over shrinkage of GLars-type estimator. We distinguish two variants of the GVISA algorithm, each one is associated with each version of GLars (I and II). Moreover, we provide a path algorithm, similar to GLars, for efficiently computing the entire sample path of GVISA coefficients. We establish a theoretical property on sparsity inequality of GVISA estimator that is a non-asymptotic bound on the estimation error. A detailed simulation study in small and high dimensional settings is performed, which illustrates the advantages of the new approach in relation to several other possible methods. Finally, we apply GVISA on two real data sets. Copyright Springer-Verlag Berlin Heidelberg 2015

Suggested Citation

  • Abdallah Mkhadri & Mohamed Ouhourane, 2015. "A group VISA algorithm for variable selection," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 24(1), pages 41-60, March.
  • Handle: RePEc:spr:stmapp:v:24:y:2015:i:1:p:41-60
    DOI: 10.1007/s10260-014-0281-8
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s10260-014-0281-8
    Download Restriction: Access to full text is restricted to subscribers.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lukas Meier & Sara Van De Geer & Peter Bühlmann, 2008. "The group lasso for logistic regression," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(1), pages 53-71, February.
    2. Meinshausen, Nicolai, 2007. "Relaxed Lasso," Computational Statistics & Data Analysis, Elsevier, vol. 52(1), pages 374-393, September.
    3. Jian Huang & Shuange Ma & Huiliang Xie & Cun-Hui Zhang, 2009. "A group bridge approach for variable selection," Biometrika, Biometrika Trust, vol. 96(2), pages 339-355.
    4. Wei, Fengrong & Zhu, Hongxiao, 2012. "Group coordinate descent algorithms for nonconvex penalized regression," Computational Statistics & Data Analysis, Elsevier, vol. 56(2), pages 316-326.
    5. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    6. Mkhadri, Abdallah & Ouhourane, Mohamed, 2013. "An extended variable inclusion and shrinkage algorithm for correlated variables," Computational Statistics & Data Analysis, Elsevier, vol. 57(1), pages 631-644.
    7. She, Yiyuan, 2012. "An iterative algorithm for fitting nonconvex penalized generalized linear models with grouped predictors," Computational Statistics & Data Analysis, Elsevier, vol. 56(10), pages 2976-2990.
    8. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    9. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    10. Ming Yuan & Yi Lin, 2006. "Model selection and estimation in regression with grouped variables," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(1), pages 49-67, February.
    Full references (including those not matched with items on IDEAS)

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stmapp:v:24:y:2015:i:1:p:41-60. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Sonal Shukla) or (Mallaigh Nolan). General contact details of provider: http://www.springer.com .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.