IDEAS home Printed from https://ideas.repec.org/p/ice/wpaper/wp35.html
   My bibliography  Save this paper

Computational Efficiency in Bayesian Model and Variable Selection

Author

Listed:
  • Jana Eklund
  • Sune Karlsson

Abstract

This paper is concerned with the efficient implementation of Bayesian model averaging (BMA) and Bayesian variable selection, when the number of candidate variables and models is large, and estimation of posterior model probabilities must be based on a subset of the models. Efficient implementation is concerned with two issues, the efficiency of the MCMC algorithm itself and efficient computation of the quantities needed to obtain a draw from the MCMC algorithm. For the first aspect, it is desirable that the chain moves well and quickly through the model space and takes draws from regions with high probabilities. In this context there is a natural trade-off between local moves, which make use of the current parameter values to propose plausible values for model parameters, and more global transitions, which potentially allow exploration of the distribution of interest in fewer steps, but where each step is more computationally intensive. We assess the convergence properties of simple samplers based on local moves and some recently proposed algorithms intended to improve on the basic samplers. For the second aspect, efficient computation within the sampler, we focus on the important case of linear models where the computations essentially reduce to least squares calculations. When the chain makes local moves, adding or dropping a variable, substantial gains in efficiency can be made by updating the previous least squares solution.

Suggested Citation

  • Jana Eklund & Sune Karlsson, 2007. "Computational Efficiency in Bayesian Model and Variable Selection," Economics wp35, Department of Economics, Central bank of Iceland.
  • Handle: RePEc:ice:wpaper:wp35
    as

    Download full text from publisher

    File URL: http://www.sedlabanki.is/lisalib/getfile.aspx?itemid=5156
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. D. G. T. Denison & B. K. Mallick & A. F. M. Smith, 1998. "Automatic Bayesian curve fitting," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 60(2), pages 333-350.
    2. Fernandez, Carmen & Ley, Eduardo & Steel, Mark F. J., 2001. "Benchmark priors for Bayesian model averaging," Journal of Econometrics, Elsevier, vol. 100(2), pages 381-427, February.
    3. Smith, Michael & Kohn, Robert, 1996. "Nonparametric regression using Bayesian variable selection," Journal of Econometrics, Elsevier, vol. 75(2), pages 317-343, December.
    4. Gary Koop & Simon Potter, 2004. "Forecasting in dynamic factor models using Bayesian model averaging," Econometrics Journal, Royal Economic Society, vol. 7(2), pages 550-565, December.
    5. Sune Karlsson & Tor Jacobson, 2004. "Finding good predictors for inflation: a Bayesian model averaging approach," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(7), pages 479-496.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jesús Crespo Cuaresma & Gernot Doppelhofer & Martin Feldkircher, 2014. "The Determinants of Economic Growth in European Regions," Regional Studies, Taylor & Francis Journals, vol. 48(1), pages 44-67, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pena, Daniel & Redondas, Dolores, 2006. "Bayesian curve estimation by model averaging," Computational Statistics & Data Analysis, Elsevier, vol. 50(3), pages 688-709, February.
    2. Eklund, Jana & Karlsson, Sune, 2007. "An Embarrassment of Riches: Forecasting Using Large Panels," Working Papers 2007:1, Örebro University, School of Business.
    3. Dimitris Korobilis, 2008. "Forecasting in vector autoregressions with many predictors," Advances in Econometrics, in: Bayesian Econometrics, pages 403-431, Emerald Group Publishing Limited.
    4. Jana Eklund & Sune Karlsson, 2007. "Forecast Combination and Model Averaging Using Predictive Measures," Econometric Reviews, Taylor & Francis Journals, vol. 26(2-4), pages 329-363.
    5. Todd E. Clark & Michael W. McCracken, 2009. "Combining Forecasts from Nested Models," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 71(3), pages 303-329, June.
    6. Michael K. Andersson & Sune Karlsson, 2008. "Bayesian forecast combination for VAR models," Advances in Econometrics, in: Bayesian Econometrics, pages 501-524, Emerald Group Publishing Limited.
    7. Panagiotelis, Anastasios & Smith, Michael, 2008. "Bayesian identification, selection and estimation of semiparametric functions in high-dimensional additive models," Journal of Econometrics, Elsevier, vol. 143(2), pages 291-316, April.
    8. Guarin, Alexander & Lozano, Ignacio, 2017. "Credit funding and banking fragility: A forecasting model for emerging economies," Emerging Markets Review, Elsevier, vol. 32(C), pages 168-189.
    9. Carmen Fernandez & Eduardo Ley & Mark F. J. Steel, 2001. "Model uncertainty in cross-country growth regressions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 16(5), pages 563-576.
    10. Lopresti, John, 2016. "Multiproduct firms and product scope adjustment in trade," Journal of International Economics, Elsevier, vol. 100(C), pages 160-173.
    11. Bin Jiang & Anastasios Panagiotelis & George Athanasopoulos & Rob Hyndman & Farshid Vahid, 2016. "Bayesian Rank Selection in Multivariate Regression," Monash Econometrics and Business Statistics Working Papers 6/16, Monash University, Department of Econometrics and Business Statistics.
    12. Villani, Mattias & Kohn, Robert & Giordani, Paolo, 2009. "Regression density estimation using smooth adaptive Gaussian mixtures," Journal of Econometrics, Elsevier, vol. 153(2), pages 155-173, December.
    13. Gilles Celeux & Mohammed El Anbari & Jean-Michel Marin & Christian P. Robert, 2010. "Regularization in Regression : Comparing Bayesian and Frequentist Methods in a Poorly Informative Situation," Working Papers 2010-43, Center for Research in Economics and Statistics.
    14. Korobilis, Dimitris, 2013. "Hierarchical shrinkage priors for dynamic regressions with many predictors," International Journal of Forecasting, Elsevier, vol. 29(1), pages 43-59.
    15. M. P. Wand, 2000. "A Comparison of Regression Spline Smoothing Procedures," Computational Statistics, Springer, vol. 15(4), pages 443-462, December.
    16. Min Wang & Xiaoqian Sun & Tao Lu, 2015. "Bayesian structured variable selection in linear regression models," Computational Statistics, Springer, vol. 30(1), pages 205-229, March.
    17. Kim, Hyun Hak & Swanson, Norman R., 2014. "Forecasting financial and macroeconomic variables using data reduction methods: New empirical evidence," Journal of Econometrics, Elsevier, vol. 178(P2), pages 352-367.
    18. Smith, Michael & Kohn, Robert, 2000. "Nonparametric seemingly unrelated regression," Journal of Econometrics, Elsevier, vol. 98(2), pages 257-281, October.
    19. Yu Yue & Paul Speckman & Dongchu Sun, 2012. "Priors for Bayesian adaptive spline smoothing," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 64(3), pages 577-613, June.
    20. Eliana González, 2010. "Bayesian Model Averaging. An Application to Forecast Inflation in Colombia," Borradores de Economia 7013, Banco de la Republica.

    More about this item

    JEL classification:

    • C11 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Bayesian Analysis: General
    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ice:wpaper:wp35. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Central Bank of Iceland (email available below). General contact details of provider: https://edirc.repec.org/data/sedgvis.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.