IDEAS home Printed from https://ideas.repec.org/a/taf/jnlasa/v110y2015i510p795-809.html
   My bibliography  Save this article

Scalable Bayesian Model Averaging Through Local Information Propagation

Author

Listed:
  • Li Ma

Abstract

This article shows that a probabilistic version of the classical forward-stepwise variable inclusion procedure can serve as a general data-augmentation scheme for model space distributions in (generalized) linear models. This latent variable representation takes the form of a Markov process, thereby allowing information propagation algorithms to be applied for sampling from model space posteriors. In particular, We propose a sequential Monte Carlo method for achieving effective unbiased Bayesian model averaging in high-dimensional problems, using proposal distributions constructed using local information propagation. The method--called LIPS for local information propagation based sampling--is illustrated using real and simulated examples with dimensionality ranging from 15 to 1000, and its performance in estimating posterior inclusion probabilities and in out-of-sample prediction is compared to those of several other methods--namely, MCMC, BAS, iBMA, and LASSO. In addition, it is shown that the latent variable representation can also serve as a modeling tool for specifying model space priors that account for knowledge regarding model complexity and conditional inclusion relationships. Supplementary materials for this article are available online.

Suggested Citation

  • Li Ma, 2015. "Scalable Bayesian Model Averaging Through Local Information Propagation," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(510), pages 795-809, June.
    • Handle: RePEc:taf:jnlasa:v:110:y:2015:i:510:p:795-809
    DOI: 10.1080/01621459.2014.980908
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/01621459.2014.980908
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1080/01621459.2014.980908?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Pierre Del Moral & Arnaud Doucet & Ajay Jasra, 2006. "Sequential Monte Carlo samplers," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(3), pages 411-436, June.
    2. James O. Berger & German Molina, 2005. "Posterior model probabilities via path‐based pairwise priors," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 59(1), pages 3-15, February.
    3. Veronika Ročková & Edward I. George, 2014. "EMVS: The EM Approach to Bayesian Variable Selection," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(506), pages 828-846, June.
    4. Smith, Michael & Kohn, Robert, 1996. "Nonparametric regression using Bayesian variable selection," Journal of Econometrics, Elsevier, vol. 75(2), pages 317-343, December.
    5. Robert Tibshirani, 2011. "Regression shrinkage and selection via the lasso: a retrospective," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 73(3), pages 273-282, June.
    6. Nicolas Chopin, 2002. "A sequential particle filter method for static models," Biometrika, Biometrika Trust, vol. 89(3), pages 539-552, August.
    7. Fernandez, Carmen & Ley, Eduardo & Steel, Mark F. J., 2001. "Benchmark priors for Bayesian model averaging," Journal of Econometrics, Elsevier, vol. 100(2), pages 381-427, February.
    8. Ajay Jasra & David A. Stephens & Christopher C. Holmes, 2007. "Population-Based Reversible Jump Markov Chain Monte Carlo," Biometrika, Biometrika Trust, vol. 94(4), pages 787-807.
    9. Hans, Chris & Dobra, Adrian & West, Mike, 2007. "Shotgun Stochastic Search for," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 507-516, June.
    10. Liang, Feng & Paulo, Rui & Molina, German & Clyde, Merlise A. & Berger, Jim O., 2008. "Mixtures of g Priors for Bayesian Variable Selection," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 410-423, March.
    11. David J. Nott & Robert Kohn, 2005. "Adaptive sampling for Bayesian variable selection," Biometrika, Biometrika Trust, vol. 92(4), pages 747-763, December.
    12. Shi, Minghui & Dunson, David B., 2011. "Bayesian variable selection via particle stochastic search," Statistics & Probability Letters, Elsevier, vol. 81(2), pages 283-291, February.
    13. Ghosh, Joyee & Clyde, Merlise A., 2011. "Rao–Blackwellization for Bayesian Variable Selection and Model Averaging in Linear and Binary Regression: A Novel Data Augmentation Approach," Journal of the American Statistical Association, American Statistical Association, vol. 106(495), pages 1041-1052.
    14. P. J. Brown & M. Vannucci & T. Fearn, 1998. "Multivariate Bayesian variable selection and prediction," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 60(3), pages 627-641.
    15. repec:wyi:journl:002173 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dimitris Korobilis & Kenichi Shimizu, 2022. "Bayesian Approaches to Shrinkage and Sparse Estimation," Foundations and Trends(R) in Econometrics, now publishers, vol. 11(4), pages 230-354, June.
    2. Ouysse, Rachida & Kohn, Robert, 2010. "Bayesian variable selection and model averaging in the arbitrage pricing theory model," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3249-3268, December.
    3. Gilles Celeux & Mohammed El Anbari & Jean-Michel Marin & Christian P. Robert, 2010. "Regularization in Regression : Comparing Bayesian and Frequentist Methods in a Poorly Informative Situation," Working Papers 2010-43, Center for Research in Economics and Statistics.
    4. Min Wang & Xiaoqian Sun & Tao Lu, 2015. "Bayesian structured variable selection in linear regression models," Computational Statistics, Springer, vol. 30(1), pages 205-229, March.
    5. Enrico Petretto & Leonardo Bottolo & Sarah R Langley & Matthias Heinig & Chris McDermott-Roe & Rizwan Sarwar & Michal Pravenec & Norbert Hübner & Timothy J Aitman & Stuart A Cook & Sylvia Richardson, 2010. "New Insights into the Genetic Control of Gene Expression using a Bayesian Multi-tissue Approach," PLOS Computational Biology, Public Library of Science, vol. 6(4), pages 1-13, April.
    6. Joyee Ghosh & Andrew E. Ghattas, 2015. "Bayesian Variable Selection Under Collinearity," The American Statistician, Taylor & Francis Journals, vol. 69(3), pages 165-173, August.
    7. Ley, Eduardo & Steel, Mark F.J., 2012. "Mixtures of g-priors for Bayesian model averaging with economic applications," Journal of Econometrics, Elsevier, vol. 171(2), pages 251-266.
    8. Ley, Eduardo & Steel, Mark F. J., 2007. "On the effect of prior assumptions in Bayesian model averaging with applications to growth regression," Policy Research Working Paper Series 4238, The World Bank.
    9. Bernardi, Mauro & Costola, Michele, 2019. "High-dimensional sparse financial networks through a regularised regression model," SAFE Working Paper Series 244, Leibniz Institute for Financial Research SAFE.
    10. Mark F. J. Steel, 2020. "Model Averaging and Its Use in Economics," Journal of Economic Literature, American Economic Association, vol. 58(3), pages 644-719, September.
    11. Theo S. Eicher & Chris Papageorgiou & Adrian E. Raftery, 2011. "Default priors and predictive performance in Bayesian model averaging, with application to growth determinants," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(1), pages 30-55, January/F.
    12. Dimitris Korobilis, 2008. "Forecasting in vector autoregressions with many predictors," Advances in Econometrics, in: Bayesian Econometrics, pages 403-431, Emerald Group Publishing Limited.
    13. Njindan Iyke, Bernard, 2015. "Macro Determinants of the Real Exchange Rate in a Small Open Small Island Economy: Evidence from Mauritius via BMA," MPRA Paper 68968, University Library of Munich, Germany.
    14. Robert Kohn & Rachida Ouysse, 2007. "Bayesian Variable Selection of Risk Factors in the APT Model," Discussion Papers 2007-32, School of Economics, The University of New South Wales.
    15. Nadja Klein & Michael Stanley Smith, 2021. "Bayesian variable selection for non‐Gaussian responses: a marginally calibrated copula approach," Biometrics, The International Biometric Society, vol. 77(3), pages 809-823, September.
    16. Posch, Konstantin & Arbeiter, Maximilian & Pilz, Juergen, 2020. "A novel Bayesian approach for variable selection in linear regression models," Computational Statistics & Data Analysis, Elsevier, vol. 144(C).
    17. Kwon, Deukwoo & Landi, Maria Teresa & Vannucci, Marina & Issaq, Haleem J. & Prieto, DaRue & Pfeiffer, Ruth M., 2011. "An efficient stochastic search for Bayesian variable selection with high-dimensional correlated predictors," Computational Statistics & Data Analysis, Elsevier, vol. 55(10), pages 2807-2818, October.
    18. Paul Hofmarcher & Jesús Crespo Cuaresma & Bettina Grün & Kurt Hornik, 2015. "Last Night a Shrinkage Saved My Life: Economic Growth, Model Uncertainty and Correlated Regressors," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 34(2), pages 133-144, March.
    19. Emanuela Ciapanna & Marco Taboga, 2019. "Bayesian Analysis of Coefficient Instability in Dynamic Regressions," Econometrics, MDPI, vol. 7(3), pages 1-32, June.
    20. Hans, Christopher M. & Peruggia, Mario & Wang, Junyan, 2023. "Empirical Bayes Model Averaging with Influential Observations: Tuning Zellner’s g Prior for Predictive Robustness," Econometrics and Statistics, Elsevier, vol. 27(C), pages 102-119.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:jnlasa:v:110:y:2015:i:510:p:795-809. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/UASA20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.