IDEAS home Printed from https://ideas.repec.org/p/ecb/ecbwps/20202378.html
   My bibliography  Save this paper

Density forecast combinations: the real-time dimension

Author

Listed:
  • McAdam, Peter
  • Warne, Anders

Abstract

Density forecast combinations are examined in real-time using the log score to compare five methods: fixed weights, static and dynamic prediction pools, as well as Bayesian and dynamic model averaging. Since real-time data involves one vintage per time period and are subject to revisions, the chosen actuals for such comparisons typically differ from the information that can be used to compute model weights. The terms observation lag and information lag are introduced to clarify the different time shifts involved for these computations and we discuss how they influence the combination methods. We also introduce upper and lower bounds for the density forecasts, allowing us to benchmark the combination methods. The empirical study employs three DSGE models and two BVARs, where the former are variants of the Smets and Wouters model and the latter are benchmarks. The models are estimated on real-time euro area data and the forecasts cover 2001–2014, focusing on inflation and output growth. We find that some combinations are superior to the individual models for the joint and the output forecasts, mainly due to over-confident forecasts of the BVARs during the Great Recession. Combinations with limited weight variation over time and with positive weights on all models provide better forecasts than those with greater weight variation. For the inflation forecasts, the DSGE models are better overall than the BVARs and the combination methods. JEL Classification: C11, C32, C52, C53, E37

Suggested Citation

  • McAdam, Peter & Warne, Anders, 2020. "Density forecast combinations: the real-time dimension," Working Paper Series 2378, European Central Bank.
  • Handle: RePEc:ecb:ecbwps:20202378
    Note: 50336
    as

    Download full text from publisher

    File URL: https://www.ecb.europa.eu//pub/pdf/scpwps/ecb.wp2378~a9b3837817.en.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Billio, Monica & Casarin, Roberto & Ravazzolo, Francesco & van Dijk, Herman K., 2013. "Time-varying combinations of predictive densities using nonlinear filtering," Journal of Econometrics, Elsevier, vol. 177(2), pages 213-232.
    2. Kapetanios, G. & Mitchell, J. & Price, S. & Fawcett, N., 2015. "Generalised density forecast combinations," Journal of Econometrics, Elsevier, vol. 188(1), pages 150-165.
    3. Acemoglu, Daron & Woodford, Michael (ed.), 2012. "NBER Macroeconomics Annual 2011," National Bureau of Economic Research Books, University of Chicago Press, number 9780226002149, July.
    4. Anne Sofie Jore & James Mitchell & Shaun P. Vahey, 2010. "Combining forecast densities from VARs with uncertain instabilities," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 621-634.
    5. Jordi Galí & Frank Smets & Rafael Wouters, 2012. "Unemployment in an Estimated New Keynesian Model," NBER Macroeconomics Annual, University of Chicago Press, vol. 26(1), pages 329-360.
    6. Michael Cai & Marco Del Negro & Edward Herbst & Ethan Matlin & Reca Sarfati & Frank Schorfheide, 2021. "Online estimation of DSGE models," The Econometrics Journal, Royal Economic Society, vol. 24(1), pages 33-58.
    7. Anders Warne & Günter Coenen & Kai Christoffel, 2017. "Marginalized Predictive Likelihood Comparisons of Linear Gaussian State‐Space Models with Applications to DSGE, DSGE‐VAR, and VAR Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(1), pages 103-119, January.
    8. Bernanke, Ben S. & Gertler, Mark & Gilchrist, Simon, 1999. "The financial accelerator in a quantitative business cycle framework," Handbook of Macroeconomics, in: J. B. Taylor & M. Woodford (ed.), Handbook of Macroeconomics, edition 1, volume 1, chapter 21, pages 1341-1393, Elsevier.
    9. Christopher A. Sims, 1993. "A Nine-Variable Probabilistic Macroeconomic Forecasting Model," NBER Chapters, in: Business Cycles, Indicators, and Forecasting, pages 179-212, National Bureau of Economic Research, Inc.
    10. John Geweke & Gianni Amisano, 2012. "Prediction with Misspecified Models," American Economic Review, American Economic Association, vol. 102(3), pages 482-486, May.
    11. Geweke, John & Amisano, Gianni, 2011. "Optimal prediction pools," Journal of Econometrics, Elsevier, vol. 164(1), pages 130-141, September.
    12. Amisano, Gianni & Giacomini, Raffaella, 2007. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests," Journal of Business & Economic Statistics, American Statistical Association, vol. 25, pages 177-190, April.
    13. Marco Del Negro & Marc P. Giannoni & Frank Schorfheide, 2015. "Inflation in the Great Recession and New Keynesian Models," American Economic Journal: Macroeconomics, American Economic Association, vol. 7(1), pages 168-196, January.
    14. Bańbura, Marta & Giannone, Domenico & Lenza, Michele, 2015. "Conditional forecasts and scenario analysis with vector autoregressions for large cross-sections," International Journal of Forecasting, Elsevier, vol. 31(3), pages 739-756.
    15. Marta Banbura & Domenico Giannone & Lucrezia Reichlin, 2010. "Large Bayesian vector auto regressions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(1), pages 71-92.
    16. Domenico Giannone & Michele Lenza & Giorgio E. Primiceri, 2015. "Prior Selection for Vector Autoregressions," The Review of Economics and Statistics, MIT Press, vol. 97(2), pages 436-451, May.
    17. Negro, Marco Del & Schorfheide, Frank, 2013. "DSGE Model-Based Forecasting," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 57-140, Elsevier.
    18. Sims, Christopher A & Zha, Tao, 1998. "Bayesian Methods for Dynamic Multivariate Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 949-968, November.
    19. Gianni Amisano & John Geweke, 2017. "Prediction Using Several Macroeconomic Models," The Review of Economics and Statistics, MIT Press, vol. 99(5), pages 912-925, December.
    20. Walter R. Gilks & Carlo Berzuini, 2001. "Following a moving target—Monte Carlo inference for dynamic Bayesian models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(1), pages 127-146.
    21. Gary Koop & Dimitris Korobilis, 2012. "Forecasting Inflation Using Dynamic Model Averaging," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 53(3), pages 867-886, August.
    22. Domenico Giannone & Michele Lenza & Giorgio E. Primiceri, 2019. "Priors for the Long Run," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(526), pages 565-580, April.
    23. Frank Smets & Rafael Wouters, 2007. "Shocks and Frictions in US Business Cycles: A Bayesian DSGE Approach," American Economic Review, American Economic Association, vol. 97(3), pages 586-606, June.
    24. Anne Opschoor & Dick van Dijk & Michel van der Wel, 2017. "Combining density forecasts using focused scoring rules," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(7), pages 1298-1313, November.
    25. Dean Croushore, 2011. "Frontiers of Real-Time Data Analysis," Journal of Economic Literature, American Economic Association, vol. 49(1), pages 72-100, March.
    26. Todd E. Clark, 2011. "Real-Time Density Forecasts From Bayesian Vector Autoregressions With Stochastic Volatility," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 29(3), pages 327-341, July.
    27. Edward P. Herbst & Frank Schorfheide, 2016. "Bayesian Estimation of DSGE Models," Economics Books, Princeton University Press, edition 1, number 10612.
    28. Newey, Whitney & West, Kenneth, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 33(1), pages 125-132.
    29. Coenen, Günter & Karadi, Peter & Schmidt, Sebastian & Warne, Anders, 2018. "The New Area-Wide Model II: an extended version of the ECB’s micro-founded model for forecasting and policy analysis with a financial sector," Working Paper Series 2200, European Central Bank.
    30. Domenico Giannone & Jérôme Henry & Magdalena Lalik & Michele Modugno, 2012. "An Area-Wide Real-Time Database for the Euro Area," The Review of Economics and Statistics, MIT Press, vol. 94(4), pages 1000-1013, November.
    31. Croushore, Dean & Stark, Tom, 2001. "A real-time data set for macroeconomists," Journal of Econometrics, Elsevier, vol. 105(1), pages 111-130, November.
    32. Waggoner, Daniel F. & Zha, Tao, 2012. "Confronting model misspecification in macroeconomics," Journal of Econometrics, Elsevier, vol. 171(2), pages 167-184.
    33. repec:hal:journl:peer-00834423 is not listed on IDEAS
    34. McAdam, Peter & Warne, Anders, 2019. "Euro area real-time density forecasting with financial or labor market frictions," International Journal of Forecasting, Elsevier, vol. 35(2), pages 580-600.
    35. Durbin, James & Koopman, Siem Jan, 2012. "Time Series Analysis by State Space Methods," OUP Catalogue, Oxford University Press, edition 2, number 9780199641178.
    36. McAlinn, Kenichiro & West, Mike, 2019. "Dynamic Bayesian predictive synthesis in time series forecasting," Journal of Econometrics, Elsevier, vol. 210(1), pages 155-169.
    37. Marta Bańbura, 2008. "Large Bayesian VARs," 2008 Meeting Papers 334, Society for Economic Dynamics.
    38. Del Negro, Marco & Hasegawa, Raiden B. & Schorfheide, Frank, 2016. "Dynamic prediction pools: An investigation of financial frictions and forecasting performance," Journal of Econometrics, Elsevier, vol. 192(2), pages 391-405.
    39. Hall, Stephen G. & Mitchell, James, 2007. "Combining density forecasts," International Journal of Forecasting, Elsevier, vol. 23(1), pages 1-13.
    40. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    41. Jana Eklund & Sune Karlsson, 2007. "Forecast Combination and Model Averaging Using Predictive Measures," Econometric Reviews, Taylor & Francis Journals, vol. 26(2-4), pages 329-363.
    42. Stark, Tom & Croushore, Dean, 2002. "Forecasting with a real-time data set for macroeconomists," Journal of Macroeconomics, Elsevier, vol. 24(4), pages 507-531, December.
    43. John Geweke, 2010. "Complete and Incomplete Econometric Models," Economics Books, Princeton University Press, edition 1, number 9218.
    44. Diks, Cees & Panchenko, Valentyn & van Dijk, Dick, 2011. "Likelihood-based scoring rules for comparing density forecasts in tails," Journal of Econometrics, Elsevier, vol. 163(2), pages 215-230, August.
    45. James H. Stock & Mark W. Watson, 1993. "Business Cycles, Indicators, and Forecasting," NBER Books, National Bureau of Economic Research, Inc, number stoc93-1, March.
    46. Sims, Christopher A., 2000. "Using a likelihood perspective to sharpen econometric discourse: Three examples," Journal of Econometrics, Elsevier, vol. 95(2), pages 443-462, April.
    47. Stock, James H. & Watson, Mark W. (ed.), 1993. "Business Cycles, Indicators, and Forecasting," National Bureau of Economic Research Books, University of Chicago Press, edition 1, number 9780226774886, December.
    48. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    49. Pauwels, Laurent L. & Vasnev, Andrey L., 2016. "A note on the estimation of optimal weights for density forecast combinations," International Journal of Forecasting, Elsevier, vol. 32(2), pages 391-397.
    50. G. Elliott & C. Granger & A. Timmermann (ed.), 2013. "Handbook of Economic Forecasting," Handbook of Economic Forecasting, Elsevier, edition 1, volume 2, number 2.
    51. Fagan, Gabriel & Henry, Jerome & Mestre, Ricardo, 2005. "An area-wide model for the euro area," Economic Modelling, Elsevier, vol. 22(1), pages 39-59, January.
    52. Smets, Frank & Warne, Anders & Wouters, Rafael, 2014. "Professional forecasters and real-time forecasting with a DSGE model," International Journal of Forecasting, Elsevier, vol. 30(4), pages 981-995.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Warne, Anders, 2023. "DSGE model forecasting: rational expectations vs. adaptive learning," Working Paper Series 2768, European Central Bank.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. McAdam, Peter & Warne, Anders, 2019. "Euro area real-time density forecasting with financial or labor market frictions," International Journal of Forecasting, Elsevier, vol. 35(2), pages 580-600.
    2. Warne, Anders, 2023. "DSGE model forecasting: rational expectations vs. adaptive learning," Working Paper Series 2768, European Central Bank.
    3. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    4. Barbara Rossi, 2019. "Forecasting in the presence of instabilities: How do we know whether models predict well and how to improve them," Economics Working Papers 1711, Department of Economics and Business, Universitat Pompeu Fabra, revised Jul 2021.
    5. Davide Pettenuzzo & Francesco Ravazzolo, 2016. "Optimal Portfolio Choice Under Decision‐Based Model Combinations," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(7), pages 1312-1332, November.
    6. Kenichiro McAlinn & Knut Are Aastveit & Jouchi Nakajima & Mike West, 2020. "Multivariate Bayesian Predictive Synthesis in Macroeconomic Forecasting," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 115(531), pages 1092-1110, July.
    7. Anders Warne & Günter Coenen & Kai Christoffel, 2017. "Marginalized Predictive Likelihood Comparisons of Linear Gaussian State‐Space Models with Applications to DSGE, DSGE‐VAR, and VAR Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(1), pages 103-119, January.
    8. Hasumi, Ryo & Iiboshi, Hirokuni & Matsumae, Tatsuyoshi & Nakamura, Daisuke, 2019. "Does a financial accelerator improve forecasts during financial crises? Evidence from Japan with prediction-pooling methods," Journal of Asian Economics, Elsevier, vol. 60(C), pages 45-68.
    9. Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.
    10. Čapek, Jan & Crespo Cuaresma, Jesús & Hauzenberger, Niko & Reichel, Vlastimil, 2023. "Macroeconomic forecasting in the euro area using predictive combinations of DSGE models," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1820-1838.
    11. Berg, Tim O. & Henzel, Steffen R., 2015. "Point and density forecasts for the euro area using Bayesian VARs," International Journal of Forecasting, Elsevier, vol. 31(4), pages 1067-1095.
    12. Gael M. Martin & David T. Frazier & Ruben Loaiza-Maya & Florian Huber & Gary Koop & John Maheu & Didier Nibbering & Anastasios Panagiotelis, 2023. "Bayesian Forecasting in the 21st Century: A Modern Review," Monash Econometrics and Business Statistics Working Papers 1/23, Monash University, Department of Econometrics and Business Statistics.
    13. Andrea Carriero & Todd E. Clark & Massimiliano Marcellino, 2016. "Common Drifting Volatility in Large Bayesian VARs," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(3), pages 375-390, July.
    14. Knut Are Aastveit & Andrea Carriero & Todd E. Clark & Massimiliano Marcellino, 2017. "Have Standard VARS Remained Stable Since the Crisis?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(5), pages 931-951, August.
    15. Del Negro, Marco & Hasegawa, Raiden B. & Schorfheide, Frank, 2016. "Dynamic prediction pools: An investigation of financial frictions and forecasting performance," Journal of Econometrics, Elsevier, vol. 192(2), pages 391-405.
    16. Gael M. Martin & David T. Frazier & Worapree Maneesoonthorn & Ruben Loaiza-Maya & Florian Huber & Gary Koop & John Maheu & Didier Nibbering & Anastasios Panagiotelis, 2022. "Bayesian Forecasting in Economics and Finance: A Modern Review," Papers 2212.03471, arXiv.org, revised Jul 2023.
    17. Knut Are Aastveit & James Mitchell & Francesco Ravazzolo & Herman van Dijk, 2018. "The Evolution of Forecast Density Combinations in Economics," Tinbergen Institute Discussion Papers 18-069/III, Tinbergen Institute.
    18. Smets, Frank & Warne, Anders & Wouters, Rafael, 2014. "Professional forecasters and real-time forecasting with a DSGE model," International Journal of Forecasting, Elsevier, vol. 30(4), pages 981-995.
    19. Bańbura, Marta & Brenna, Federica & Paredes, Joan & Ravazzolo, Francesco, 2021. "Combining Bayesian VARs with survey density forecasts: does it pay off?," Working Paper Series 2543, European Central Bank.
    20. Miranda-Agrippino, Silvia & Ricco, Giovanni, 2018. "Bayesian Vector Autoregressions," The Warwick Economics Research Paper Series (TWERPS) 1159, University of Warwick, Department of Economics.

    More about this item

    Keywords

    Bayesian inference; euro area; forecast comparisons; model averaging; prediction pools; predictive likelihood;
    All these keywords.

    JEL classification:

    • C11 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Bayesian Analysis: General
    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • E37 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Forecasting and Simulation: Models and Applications

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ecb:ecbwps:20202378. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Official Publications (email available below). General contact details of provider: https://edirc.repec.org/data/emieude.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.