IDEAS home Printed from https://ideas.repec.org/a/eee/econom/v167y2012i2p358-369.html
   My bibliography  Save this article

Bayesian averaging, prediction and nonnested model selection

Author

Listed:
  • Hong, Han
  • Preston, Bruce

Abstract

This paper studies the asymptotic relationship between Bayesian model averaging and post-selection frequentist predictors in both nested and nonnested models. We derive conditions under which their difference is of a smaller order of magnitude than the inverse of the square root of the sample size in large samples. This result depends crucially on the relation between posterior odds and frequentist model selection criteria. Weak conditions are given under which consistent model selection is feasible, regardless of whether models are nested or nonnested and regardless of whether models are correctly specified or not, in the sense that they select the best model with the least number of parameters with probability converging to 1. Under these conditions, Bayesian posterior odds and BICs are consistent for selecting among nested models, but are not consistent for selecting among nonnested models and possibly overlapping models. These findings have important bearing for applied researchers who are frequent users of model selection tools for empirical investigation of model predictions.

Suggested Citation

  • Hong, Han & Preston, Bruce, 2012. "Bayesian averaging, prediction and nonnested model selection," Journal of Econometrics, Elsevier, vol. 167(2), pages 358-369.
  • Handle: RePEc:eee:econom:v:167:y:2012:i:2:p:358-369
    DOI: 10.1016/j.jeconom.2011.09.021
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304407611002028
    Download Restriction: Full text for ScienceDirect subscribers only

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Jesús Fernández-Villaverde & Juan F. Rubio-Ramírez, 2007. "Estimating Macroeconomic Models: A Likelihood Approach," Review of Economic Studies, Oxford University Press, vol. 74(4), pages 1059-1087.
    2. Hansen, Lars Peter, 1982. "Large Sample Properties of Generalized Method of Moments Estimators," Econometrica, Econometric Society, vol. 50(4), pages 1029-1054, July.
    3. Orazio P. Attanasio & Hamish Low, 2004. "Estimating Euler Equations," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 7(2), pages 405-435, April.
    4. repec:fth:jonhop:390 is not listed on IDEAS
    5. Hong, Han & Preston, Bruce & Shum, Matthew, 2003. "Generalized Empirical Likelihood Based Model Selection Criteria For Moment Condition Models," Econometric Theory, Cambridge University Press, vol. 19(06), pages 923-943, December.
    6. Sin, Chor-Yiu & White, Halbert, 1996. "Information criteria for selecting possibly misspecified parametric models," Journal of Econometrics, Elsevier, vol. 71(1-2), pages 207-225.
    7. Gourieroux,Christian & Monfort,Alain, 1995. "Statistics and Econometric Models," Cambridge Books, Cambridge University Press, number 9780521471626, April.
    8. Carroll Christopher Dixon, 2001. "Death to the Log-Linearized Consumption Euler Equation! (And Very Poor Health to the Second-Order Approximation)," The B.E. Journal of Macroeconomics, De Gruyter, vol. 1(1), pages 1-38, April.
    9. Juan F. Rubio-Ramirez & Jesus Fernández-Villaverde, 2005. "Estimating dynamic equilibrium economies: linear versus nonlinear likelihood," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(7), pages 891-910.
    10. Todd E. Clark & Michael W. McCracken, 2009. "Combining Forecasts from Nested Models," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 71(3), pages 303-329, June.
    11. Fan, Yanqin, 1994. "Testing the Goodness of Fit of a Parametric Density Function by Kernel Method," Econometric Theory, Cambridge University Press, vol. 10(02), pages 316-356, June.
    12. Powell, James L., 1986. "Censored regression quantiles," Journal of Econometrics, Elsevier, vol. 32(1), pages 143-155, June.
    13. Chernozhukov, Victor & Hong, Han, 2003. "An MCMC approach to classical estimation," Journal of Econometrics, Elsevier, vol. 115(2), pages 293-346, August.
    14. Victor Chernozhukov & Christian Hansen, 2005. "An IV Model of Quantile Treatment Effects," Econometrica, Econometric Society, vol. 73(1), pages 245-261, January.
    15. Lubik, Thomas A. & Schorfheide, Frank, 2007. "Do central banks respond to exchange rate movements? A structural investigation," Journal of Monetary Economics, Elsevier, vol. 54(4), pages 1069-1087, May.
    16. Vuong, Quang H, 1989. "Likelihood Ratio Tests for Model Selection and Non-nested Hypotheses," Econometrica, Econometric Society, vol. 57(2), pages 307-333, March.
    17. Wright, Jonathan H., 2008. "Bayesian Model Averaging and exchange rate forecasts," Journal of Econometrics, Elsevier, vol. 146(2), pages 329-341, October.
    18. Frank Schorfheide, 2000. "Loss function-based evaluation of DSGE models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 15(6), pages 645-670.
    19. Pakes, Ariel & Pollard, David, 1989. "Simulation and the Asymptotics of Optimization Estimators," Econometrica, Econometric Society, vol. 57(5), pages 1027-1057, September.
    20. Newey, Whitney K, 1985. "Maximum Likelihood Specification Testing and Conditional Moment Tests," Econometrica, Econometric Society, vol. 53(5), pages 1047-1070, September.
    21. Berry, Steven & Levinsohn, James & Pakes, Ariel, 1995. "Automobile Prices in Market Equilibrium," Econometrica, Econometric Society, vol. 63(4), pages 841-890, July.
    22. Jesús Fernández-Villaverde & Juan Francisco Rubio-Ramírez, 2004. "Estimating nonlinear dynamic equilibrium economies: a likelihood approach," FRB Atlanta Working Paper 2004-1, Federal Reserve Bank of Atlanta.
    23. Justiniano, Alejandro & Preston, Bruce, 2010. "Can structural small open-economy models account for the influence of foreign disturbances?," Journal of International Economics, Elsevier, vol. 81(1), pages 61-74, May.
    24. Timmermann, Allan, 2006. "Forecast Combinations," Handbook of Economic Forecasting, Elsevier.
    25. James H. Stock & Mark W.Watson, 2003. "Forecasting Output and Inflation: The Role of Asset Prices," Journal of Economic Literature, American Economic Association, vol. 41(3), pages 788-829, September.
    26. Andrews, Donald W. K. & Lu, Biao, 2001. "Consistent model and moment selection procedures for GMM estimation with application to dynamic panel data models," Journal of Econometrics, Elsevier, vol. 101(1), pages 123-164, March.
    27. Mark W. Watson & James H. Stock, 2004. "Combination forecasts of output growth in a seven-country data set," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(6), pages 405-430.
    28. Pötscher, B.M., 1991. "Effects of Model Selection on Inference," Econometric Theory, Cambridge University Press, vol. 7(02), pages 163-185, June.
    29. Fernandez-Villaverde, Jesus & Francisco Rubio-Ramirez, Juan, 2004. "Comparing dynamic equilibrium models to data: a Bayesian approach," Journal of Econometrics, Elsevier, vol. 123(1), pages 153-187, November.
    30. Pollard, David, 1991. "Asymptotics for Least Absolute Deviation Regression Estimators," Econometric Theory, Cambridge University Press, vol. 7(02), pages 186-199, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Li, Cheng & Jiang, Wenxin, 2016. "On oracle property and asymptotic validity of Bayesian generalized method of moments," Journal of Multivariate Analysis, Elsevier, vol. 145(C), pages 132-147.
    2. Erhard Reschenhofer & David Preinerstorfer & Lukas Steinberger, 2013. "Non-monotonic penalizing for the number of structural breaks," Computational Statistics, Springer, vol. 28(6), pages 2585-2598, December.
    3. Mao, Guangyu, 2015. "Model selection of M-estimation models using least squares approximation," Statistics & Probability Letters, Elsevier, vol. 99(C), pages 238-243.
    4. Mao, Guangyu, 2013. "Model selection for regression with heteroskedastic and autocorrelated errors," Economics Letters, Elsevier, vol. 118(3), pages 497-501.
    5. Alin T Mirestean & Charalambos G Tsangarides & Huigang Chen, 2009. "Limited Information Bayesian Model Averaging for Dynamic Panels with Short Time Periods," IMF Working Papers 09/74, International Monetary Fund.
    6. Huigang Chen & Alin T Mirestean & Charalambos G Tsangarides, 2011. "Limited Information Bayesian Model Averaging for Dynamic Panels with An Application to a Trade Gravity Model," IMF Working Papers 11/230, International Monetary Fund.
    7. Javier García - Cicco & Roque Montero, 2011. "Modeling Copper Price: A Regime-Switching Approach," Working Papers Central Bank of Chile 613, Central Bank of Chile.
    8. Koetter, Michael & Poghosyan, Tigran, 2010. "Real estate prices and bank stability," Journal of Banking & Finance, Elsevier, vol. 34(6), pages 1129-1138, June.

    More about this item

    Keywords

    Model selection criteria; Nonnested; Posterior odds; BIC;

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:167:y:2012:i:2:p:358-369. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Dana Niculescu). General contact details of provider: http://www.elsevier.com/locate/jeconom .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.