IDEAS home Printed from https://ideas.repec.org/a/spr/compst/v41y2026i3d10.1007_s00180-026-01733-7.html

Variational inference for sparse poisson regression

Author

Listed:
  • Mitra Kharabati

    (University of Tehran, Department of Statistics, School of Mathematics, Statistics, and Computer Science, College of Science)

  • Morteza Amini

    (University of Tehran, Department of Statistics, School of Mathematics, Statistics, and Computer Science, College of Science)

  • Mohammad Arashi

    (Ferdowsi University of Mashhad, Department of Statistics, Faculty of Mathematical Sciences)

Abstract

We have utilized the non-conjugate Variational Bayesian (VB) method for the problem of the sparse Poisson regression model. To provide approximate conjugacy in the model, the likelihood is approximated by a quadratic function, yielding conjugacy between the approximation component and the Gaussian prior on the regression coefficient. Three sparsity-enforcing priors (Laplace, Continuous Spike and Slab, and Bernoulli) are used for this problem. The proposed models are compared with each other, the associated MCMC models, and two frequentist sparse Poisson methods (LASSO and SCAD) to evaluate their estimation, prediction, and sparsity performance. In a simulation study, the proposed VB methods closely approximate the posterior parameter distribution while achieving significantly faster computation than benchmark MCMC methods. Using several benchmark count response data sets, the prediction performance of the proposed methods is evaluated in real-world applications.

Suggested Citation

  • Mitra Kharabati & Morteza Amini & Mohammad Arashi, 2026. "Variational inference for sparse poisson regression," Computational Statistics, Springer, vol. 41(3), pages 1-50, April.
    • Handle: RePEc:spr:compst:v:41:y:2026:i:3:d:10.1007_s00180-026-01733-7
    DOI: 10.1007/s00180-026-01733-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00180-026-01733-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00180-026-01733-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Sungduk Kim & Zhen Chen & Zhiwei Zhang & Bruce G. Simons-Morton & Paul S. Albert, 2013. "Bayesian Hierarchical Poisson Regression Models: An Application to a Driving Study With Kinematic Events," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 108(502), pages 494-503, June.
    2. David M. Blei & Alp Kucukelbir & Jon D. McAuliffe, 2017. "Variational Inference: A Review for Statisticians," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(518), pages 859-877, April.
    3. W. K. Hastings, 1970. "Monte Carlo sampling methods using Markov chains and their applications," Biometrika, Biometrika Trust, vol. 57(1), pages 97-109.
    4. Feng-Chang Xie & Jin-Guan Lin & Bo-Cheng Wei, 2014. "Bayesian zero-inflated generalized Poisson regression model: estimation and case influence diagnostics," Journal of Applied Statistics, Taylor & Francis Journals, vol. 41(6), pages 1383-1392, June.
    5. Minsuk Shin & Jun S. Liu, 2022. "Neuronized Priors for Bayesian Sparse Linear Regression," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 117(540), pages 1695-1710, October.
    6. Håvard Rue & Sara Martino & Nicolas Chopin, 2009. "Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 71(2), pages 319-392, April.
    7. Howard D. Bondell & Brian J. Reich, 2012. "Consistent High-Dimensional Bayesian Variable Selection via Penalized Credible Regions," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(500), pages 1610-1624, December.
    8. Fair, Ray C, 1978. "A Theory of Extramarital Affairs," Journal of Political Economy, University of Chicago Press, vol. 86(1), pages 45-61, February.
    9. Zhang, Chun-Xia & Xu, Shuang & Zhang, Jiang-She, 2019. "A novel variational Bayesian method for variable selection in logistic regression models," Computational Statistics & Data Analysis, Elsevier, vol. 133(C), pages 1-19.
    10. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    11. Powers, Stephanie & Gerlach, Richard & Stamey, James, 2010. "Bayesian variable selection for Poisson regression with underreported responses," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3289-3299, December.
    12. Kolyan Ray & Botond Szabó, 2022. "Variational Bayes for High-Dimensional Linear Regression With Sparse Priors," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 117(539), pages 1270-1281, September.
    13. Hiroki Saishu & Kota Kudo & Yuichi Takano, 2021. "Sparse Poisson regression via mixed-integer optimization," PLOS ONE, Public Library of Science, vol. 16(4), pages 1-17, April.
    14. Seonghyun Jeong & Subhashis Ghosal, 2021. "Posterior contraction in sparse generalized linear models [Model selection and minimax estimation in generalized linear models]," Biometrika, Biometrika Trust, vol. 108(2), pages 367-379.
    15. Ming-Hui Chen & Joseph G. Ibrahim, 2000. "Bayesian Predictive Inference for Time Series Count Data," Biometrics, The International Biometric Society, vol. 56(3), pages 678-685, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Juanjuan Zhang & Weixian Wang & Mingming Yang & Maozai Tian, 2025. "Variational Bayesian Variable Selection in Logistic Regression Based on Spike-and-Slab Lasso," Mathematics, MDPI, vol. 13(13), pages 1-18, July.
    2. McLain, Alexander C. & Zgodic, Anja & Bondell, Howard, 2025. "Efficient sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithm," Computational Statistics & Data Analysis, Elsevier, vol. 207(C).
    3. Ander Wilson & Brian J. Reich, 2014. "Confounder selection via penalized credible regions," Biometrics, The International Biometric Society, vol. 70(4), pages 852-861, December.
    4. Soutik Ghosal & Timothy S. Lau & Jeremy Gaskins & Maiying Kong, 2020. "A hierarchical mixed effect hurdle model for spatiotemporal count data and its application to identifying factors impacting health professional shortages," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 69(5), pages 1121-1144, November.
    5. Jianhua Zhao & Changchun Shang & Shulan Li & Ling Xin & Philip L. H. Yu, 2025. "Choosing the number of factors in factor analysis with incomplete data via a novel hierarchical Bayesian information criterion," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 19(1), pages 209-235, March.
    6. Qin, Shanshan & Zhang, Guanlin & Wu, Yuehua & Zhu, Zhongyi, 2025. "Bayesian grouping-Gibbs sampling estimation of high-dimensional linear model with non-sparsity," Computational Statistics & Data Analysis, Elsevier, vol. 203(C).
    7. Lu, Zeng-Hua, 2009. "Covariate selection in mixture models with the censored response variable," Computational Statistics & Data Analysis, Elsevier, vol. 53(7), pages 2710-2723, May.
    8. Dang, Khue-Dung & Quiroz, Matias & Kohn, Robert & Tran, Minh-Ngoc & Villani, Mattias, 2019. "Hamiltonian Monte Carlo with Energy Conserving Subsampling," Working Paper Series 372, Sveriges Riksbank (Central Bank of Sweden).
    9. Zhiqiang Wang & Ying Wu, 2025. "Gaussian variational approximation for Bayesian Lasso quantile regression model with zero-or-one inflated proportional data," Computational Statistics, Springer, vol. 40(8), pages 4853-4874, November.
    10. Fadel Hamid Hadi ALHUSSEINI, 2017. "New Bayesian Lasso in Tobit Quantile Regression," Romanian Statistical Review Supplement, Romanian Statistical Review, vol. 65(6), pages 213-229, June.
    11. David Rossell & Oriol Abril & Anirban Bhattacharya, 2021. "Approximate Laplace approximations for scalable model selection," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(4), pages 853-879, September.
    12. Zhang, Chun-Xia & Xu, Shuang & Zhang, Jiang-She, 2019. "A novel variational Bayesian method for variable selection in logistic regression models," Computational Statistics & Data Analysis, Elsevier, vol. 133(C), pages 1-19.
    13. repec:osf:socarx:sx9um_v1 is not listed on IDEAS
    14. Anna Freni-Sterrantino & Denis Rustand & Janet van Niekerk & Elias Teixeira Krainski & Håvard Rue, 2025. "A graphical framework for interpretable correlation matrix models for multivariate regression," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 34(3), pages 409-447, July.
    15. Gael M. Martin & David T. Frazier & Christian P. Robert, 2020. "Computing Bayes: Bayesian Computation from 1763 to the 21st Century," Monash Econometrics and Business Statistics Working Papers 14/20, Monash University, Department of Econometrics and Business Statistics.
    16. Weixian Wang & Maozai Tian, 2024. "Variational Bayesian EM Algorithm for Quantile Regression in Linear Mixed Effects Models," Mathematics, MDPI, vol. 12(21), pages 1-16, October.
    17. Gael M. Martin & David T. Frazier & Ruben Loaiza-Maya & Florian Huber & Gary Koop & John Maheu & Didier Nibbering & Anastasios Panagiotelis, 2023. "Bayesian Forecasting in the 21st Century: A Modern Review," Monash Econometrics and Business Statistics Working Papers 1/23, Monash University, Department of Econometrics and Business Statistics.
    18. Linda S. L. Tan, 2021. "Use of model reparametrization to improve variational Bayes," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(1), pages 30-57, February.
    19. Gael M. Martin & David T. Frazier & Christian P. Robert, 2021. "Approximating Bayes in the 21st Century," Monash Econometrics and Business Statistics Working Papers 24/21, Monash University, Department of Econometrics and Business Statistics.
    20. Murphy, James, 2020. "Explanatory Item Response Models for Dyadic Data from Multiple Groups," SocArXiv sx9um, Center for Open Science.
    21. Matias Quiroz & Mattias Villani & Robert Kohn & Minh-Ngoc Tran & Khue-Dung Dang, 2018. "Subsampling MCMC - an Introduction for the Survey Statistician," Sankhya A: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 80(1), pages 33-69, December.

    More about this item

    Keywords

    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:compst:v:41:y:2026:i:3:d:10.1007_s00180-026-01733-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.