IDEAS home Printed from
   My bibliography  Save this article

Bayes model averaging with selection of regressors


  • P. J. Brown
  • M. Vannucci
  • T. Fearn


When a number of distinct models contend for use in prediction, the choice of a single model can offer rather unstable predictions. In regression, stochastic search variable selection with Bayesian model averaging offers a cure for this robustness issue but at the expense of requiring very many predictors. Here we look at Bayes model averaging incorporating variable selection for prediction. This offers similar mean-square errors of prediction but with a vastly reduced predictor space. This can greatly aid the interpretation of the model. It also reduces the cost if measured variables have costs. The development here uses decision theory in the context of the multivariate general linear model. In passing, this reduced predictor space Bayes model averaging is contrasted with single-model approximations. A fast algorithm for updating regressions in the Markov chain Monte Carlo searches for posterior inference is developed, allowing many more variables than observations to be contemplated. We discuss the merits of absolute rather than proportionate shrinkage in regression, especially when there are more variables than observations. The methodology is illustrated on a set of spectroscopic data used for measuring the amounts of different sugars in an aqueous solution. Copyright 2002 Royal Statistical Society.

Suggested Citation

  • P. J. Brown & M. Vannucci & T. Fearn, 2002. "Bayes model averaging with selection of regressors," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(3), pages 519-536.
  • Handle: RePEc:bla:jorssb:v:64:y:2002:i:3:p:519-536

    Download full text from publisher

    File URL:
    File Function: link to full text
    Download Restriction: Access to full text is restricted to subscribers.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    1. Merlise Clyde & Edward I. George, 2000. "Flexible empirical Bayes estimation for wavelets," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 62(4), pages 681-698.
    2. Rolf Sundberg, 1999. "Multivariate Calibration - Direct and Indirect Regression Methodology," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 26(2), pages 161-207.
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Simila, Timo & Tikka, Jarkko, 2007. "Input selection and shrinkage in multiresponse linear regression," Computational Statistics & Data Analysis, Elsevier, vol. 52(1), pages 406-422, September.
    2. Theo S. Eicher & Chris Papageorgiou & Adrian E. Raftery, 2011. "Default priors and predictive performance in Bayesian model averaging, with application to growth determinants," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(1), pages 30-55, January/F.
    3. D. Fouskakis & I. Ntzoufras & D. Draper, 2009. "Population-based reversible jump Markov chain Monte Carlo methods for Bayesian variable selection and evaluation under cost limit restrictions," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 58(3), pages 383-403.
    4. Korobilis, Dimitris, 2008. "Forecasting in vector autoregressions with many predictors," MPRA Paper 21122, University Library of Munich, Germany.
    5. Chen, Kun & Jiang, Wenxin & Tanner, Martin A., 2010. "A note on some algorithms for the Gibbs posterior," Statistics & Probability Letters, Elsevier, vol. 80(15-16), pages 1234-1241, August.
    6. ter Braak, Cajo J.F., 2006. "Bayesian sigmoid shrinkage with improper variance priors and an application to wavelet denoising," Computational Statistics & Data Analysis, Elsevier, vol. 51(2), pages 1232-1242, November.
    7. Oliver J. Rutz & Michael Trusov & Randolph E. Bucklin, 2011. "Modeling Indirect Effects of Paid Search Advertising: Which Keywords Lead to More Future Visits?," Marketing Science, INFORMS, vol. 30(4), pages 646-665, July.
    8. Nott, David J. & Leng, Chenlei, 2010. "Bayesian projection approaches to variable selection in generalized linear models," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3227-3241, December.
    9. Hongxiao Zhu & Marina Vannucci & Dennis D. Cox, 2010. "A Bayesian Hierarchical Model for Classification with Selection of Functional Predictors," Biometrics, The International Biometric Society, vol. 66(2), pages 463-473, June.
    10. Steven N. Durlauf & Andros Kourtellos & Chih Ming Tan, 2005. "How Robust Are the Linkages Between Religiosity and Economic Growth," Discussion Papers Series, Department of Economics, Tufts University 0510, Department of Economics, Tufts University.
    11. Ander Wilson & Brian J. Reich, 2014. "Confounder selection via penalized credible regions," Biometrics, The International Biometric Society, vol. 70(4), pages 852-861, December.
    12. Ouysse, Rachida & Kohn, Robert, 2010. "Bayesian variable selection and model averaging in the arbitrage pricing theory model," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3249-3268, December.

    More about this item


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssb:v:64:y:2002:i:3:p:519-536. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Wiley-Blackwell Digital Licensing) or (Christopher F. Baum). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.