IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v13y2025i13p2205-d1695819.html
   My bibliography  Save this article

Variational Bayesian Variable Selection in Logistic Regression Based on Spike-and-Slab Lasso

Author

Listed:
  • Juanjuan Zhang

    (School of Digital Economy and Trade, Guangzhou Huashang College, Guangzhou 511300, China
    These authors contributed equally to this work.)

  • Weixian Wang

    (School of Mathematics and Statistics, Guangxi Normal University, Guilin 541006, China
    These authors contributed equally to this work.)

  • Mingming Yang

    (School of Tourism, Xinjiang University of Finance and Economics, Urumqi 830012, China)

  • Maozai Tian

    (School of Statistics and Data Science, Xinjiang University of Finance and Economics, Urumqi 830012, China
    Center for Applied Statistics, School of Statistics, Renmin University of China, Beijing 100872, China)

Abstract

Logistic regression is often used to solve classification problems. This article combines the advantages of Bayesian methods and spike-and-slab Lasso to select variables in high-dimensional logistic regression. The method of introducing a new hidden variable or approximating the lower bound is used to solve the problem of logistic functions without conjugate priors. The Laplace distribution in spike-and-slab Lasso is expressed as a hierarchical form of normal distribution and exponential distribution, so that all parameters in the model are posterior distributions that are easy to deal with. Considering the high time cost of parameter estimation and variable selection in high-dimensional models, we use the variational Bayesian algorithm to perform posterior inference on the parameters in the model. From the simulation results, it can be seen that it is an adaptive prior that can perform parameter estimation and variable selection well in high-dimensional logistic regression. From the perspective of algorithm running time, the method proposed in this article also has high computational efficiency in many cases.

Suggested Citation

  • Juanjuan Zhang & Weixian Wang & Mingming Yang & Maozai Tian, 2025. "Variational Bayesian Variable Selection in Logistic Regression Based on Spike-and-Slab Lasso," Mathematics, MDPI, vol. 13(13), pages 1-18, July.
    • Handle: RePEc:gam:jmathe:v:13:y:2025:i:13:p:2205-:d:1695819
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/13/13/2205/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/13/13/2205/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. David M. Blei & Alp Kucukelbir & Jon D. McAuliffe, 2017. "Variational Inference: A Review for Statisticians," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(518), pages 859-877, April.
    2. Nicholas G. Polson & James G. Scott & Jesse Windle, 2013. "Bayesian Inference for Logistic Models Using Pólya--Gamma Latent Variables," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 108(504), pages 1339-1349, December.
    3. Zhang, Chun-Xia & Xu, Shuang & Zhang, Jiang-She, 2019. "A novel variational Bayesian method for variable selection in logistic regression models," Computational Statistics & Data Analysis, Elsevier, vol. 133(C), pages 1-19.
    4. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    5. Kolyan Ray & Botond Szabó, 2022. "Variational Bayes for High-Dimensional Linear Regression With Sparse Priors," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 117(539), pages 1270-1281, September.
    6. Veronika Ročková & Edward I. George, 2018. "The Spike-and-Slab LASSO," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(521), pages 431-444, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. McLain, Alexander C. & Zgodic, Anja & Bondell, Howard, 2025. "Efficient sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithm," Computational Statistics & Data Analysis, Elsevier, vol. 207(C).
    2. Zhang, Chun-Xia & Xu, Shuang & Zhang, Jiang-She, 2019. "A novel variational Bayesian method for variable selection in logistic regression models," Computational Statistics & Data Analysis, Elsevier, vol. 133(C), pages 1-19.
    3. Ouyang, Jiarong & Cao, Xuan, 2024. "Consistent skinny Gibbs in probit regression," Computational Statistics & Data Analysis, Elsevier, vol. 198(C).
    4. Bansal, Prateek & Krueger, Rico & Graham, Daniel J., 2021. "Fast Bayesian estimation of spatial count data models," Computational Statistics & Data Analysis, Elsevier, vol. 157(C).
    5. Minerva Mukhopadhyay & David B. Dunson, 2020. "Targeted Random Projection for Prediction From High-Dimensional Features," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 115(532), pages 1998-2010, December.
    6. Jianhua Zhao & Changchun Shang & Shulan Li & Ling Xin & Philip L. H. Yu, 2025. "Choosing the number of factors in factor analysis with incomplete data via a novel hierarchical Bayesian information criterion," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 19(1), pages 209-235, March.
    7. Posch, Konstantin & Arbeiter, Maximilian & Pilz, Juergen, 2020. "A novel Bayesian approach for variable selection in linear regression models," Computational Statistics & Data Analysis, Elsevier, vol. 144(C).
    8. Tanin Sirimongkolkasem & Reza Drikvandi, 2019. "On Regularisation Methods for Analysis of High Dimensional Data," Annals of Data Science, Springer, vol. 6(4), pages 737-763, December.
    9. Mogliani, Matteo & Simoni, Anna, 2021. "Bayesian MIDAS penalized regressions: Estimation, selection, and prediction," Journal of Econometrics, Elsevier, vol. 222(1), pages 833-860.
    10. Kshitij Khare & Malay Ghosh, 2022. "MCMC Convergence for Global-Local Shrinkage Priors," Journal of Quantitative Economics, Springer;The Indian Econometric Society (TIES), vol. 20(1), pages 211-234, September.
    11. Banerjee, Sayantan, 2022. "Horseshoe shrinkage methods for Bayesian fusion estimation," Computational Statistics & Data Analysis, Elsevier, vol. 174(C).
    12. Nicholas G. Polson & James G. Scott, 2016. "Mixtures, envelopes and hierarchical duality," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(4), pages 701-727, September.
    13. Fan, Jianqing & Jiang, Bai & Sun, Qiang, 2022. "Bayesian factor-adjusted sparse regression," Journal of Econometrics, Elsevier, vol. 230(1), pages 3-19.
    14. Jieun Lee & Gyuhyeong Goh, 2024. "A hybrid deterministic–deterministic approach for high-dimensional Bayesian variable selection with a default prior," Computational Statistics, Springer, vol. 39(3), pages 1659-1681, May.
    15. Qi Zhang & Yihui Zhang & Yemao Xia, 2024. "Bayesian Feature Extraction for Two-Part Latent Variable Model with Polytomous Manifestations," Mathematics, MDPI, vol. 12(5), pages 1-23, March.
    16. Zeyu Lu & Lin Xu & Xinlei Wang, 2025. "BIT: Bayesian Identification of Transcriptional regulators from epigenomics-based query region sets," Nature Communications, Nature, vol. 16(1), pages 1-17, December.
    17. Dimitris Korobilis & Kenichi Shimizu, 2022. "Bayesian Approaches to Shrinkage and Sparse Estimation," Foundations and Trends(R) in Econometrics, now publishers, vol. 11(4), pages 230-354, June.
    18. Wang, Xiaoqing & Feng, Xiangnan & Song, Xinyuan, 2020. "Joint analysis of semicontinuous data with latent variables," Computational Statistics & Data Analysis, Elsevier, vol. 151(C).
    19. Luca Benedetti & Eric Boniardi & Leonardo Chiani & Jacopo Ghirri & Marta Mastropietro & Andrea Cappozzo & Francesco Denti, 2024. "Variational inference for semiparametric Bayesian novelty detection in large datasets," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 18(3), pages 681-703, September.
    20. Yemao Xia & Jinye Chen & Depeng Jiang, 2024. "Variational Bayesian analysis for two-part latent variable model," Computational Statistics, Springer, vol. 39(4), pages 2259-2290, June.

    More about this item

    Keywords

    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:13:y:2025:i:13:p:2205-:d:1695819. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.