IDEAS home Printed from https://ideas.repec.org/p/ecb/ecbwps/2008950.html
   My bibliography  Save this paper

Is forecasting with large models informative? Assessing the role of judgement in macroeconomic forecasts

Author

Listed:
  • McAdam, Peter
  • Mestre, Ricardo

Abstract

We evaluate residual projection strategies in the context of a large-scale macro model of the euro area and smaller benchmark time-series models. The exercises attempt to measure the accuracy of model-based forecasts simulated both out-of-sample and in-sample. Both exercises incorporate alternative residual-projection methods, to assess the importance of unaccounted-for breaks in forecast accuracy and off-model judgment. Conclusions reached are that simple mechanical residual adjustments have a significant impact of forecasting accuracy irrespective of the model in use, ostensibly due to the presence of breaks in trends in the data. The testing procedure and conclusions are applicable to a wide class of models and thus of general interest. JEL Classification: C52, E30, E32, E37

Suggested Citation

  • McAdam, Peter & Mestre, Ricardo, 2008. "Is forecasting with large models informative? Assessing the role of judgement in macroeconomic forecasts," Working Paper Series 950, European Central Bank.
  • Handle: RePEc:ecb:ecbwps:2008950
    Note: 50336
    as

    Download full text from publisher

    File URL: https://www.ecb.europa.eu//pub/pdf/scpwps/ecbwp950.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Todd Clark & Michael McCracken, 2005. "Evaluating Direct Multistep Forecasts," Econometric Reviews, Taylor & Francis Journals, vol. 24(4), pages 369-404.
    2. John F. Henry & L. Randall Wray, 1998. "Economic Time," Macroeconomics 9811004, University Library of Munich, Germany.
    3. Atsushi Inoue & Lutz Kilian, 2005. "In-Sample or Out-of-Sample Tests of Predictability: Which One Should We Use?," Econometric Reviews, Taylor & Francis Journals, vol. 23(4), pages 371-402.
    4. Marcellino, Massimiliano & Stock, James H. & Watson, Mark W., 2003. "Macroeconomic forecasting in the Euro area: Country specific versus area-wide information," European Economic Review, Elsevier, vol. 47(1), pages 1-18, February.
    5. Brock, William A. & Durlauf, Steven N. & West, Kenneth D., 2007. "Model uncertainty and policy evaluation: Some theory and empirics," Journal of Econometrics, Elsevier, vol. 136(2), pages 629-664, February.
    6. Stock, James H. & Watson, Mark W., 1999. "Business cycle fluctuations in us macroeconomic time series," Handbook of Macroeconomics, in: J. B. Taylor & M. Woodford (ed.), Handbook of Macroeconomics, edition 1, volume 1, chapter 1, pages 3-64, Elsevier.
    7. Gali, Jordi & Gertler, Mark & Lopez-Salido, J. David, 2001. "European inflation dynamics," European Economic Review, Elsevier, vol. 45(7), pages 1237-1270.
    8. Ramón Adalid & Günter Coenen & Peter McAdam & Stefano Siviero, 2005. "The Performance and Robustness of Interest-Rate Rules in Models of the Euro Area," International Journal of Central Banking, International Journal of Central Banking, vol. 1(1), May.
    9. Stock, James H & Watson, Mark W, 1996. "Evidence on Structural Instability in Macroeconomic Time Series Relations," Journal of Business & Economic Statistics, American Statistical Association, vol. 14(1), pages 11-30, January.
    10. Jushan Bai & Pierre Perron, 2003. "Computation and analysis of multiple structural change models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 18(1), pages 1-22.
    11. Stefano Siviero & Daniele Terlizzese, 2008. "Macroeconomic Forecasting: Debunking a Few Old Wives' Tales," Journal of Business Cycle Measurement and Analysis, OECD Publishing, Centre for International Research on Economic Tendency Surveys, vol. 2007(3), pages 287-316.
    12. Dieppe, Alistair & McAdam, Peter, 2006. "Monetary policy under a liquidity trap: Simulation evidence for the euro area," Journal of the Japanese and International Economies, Elsevier, vol. 20(3), pages 338-363, September.
    13. G. Coenen & J.-L. Vega, 2001. "The demand for M3 in the euro area," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 16(6), pages 727-748.
    14. Benati, Luca & Vitale, Giovanni, 2007. "Joint estimation of the natural rate of interest, the natural rate of unemployment, expected inflation, and potential output," Working Paper Series 797, European Central Bank.
    15. Christopher A. Sims, 2002. "The Role of Models and Probabilities in the Monetary Policy Process," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 33(2), pages 1-62.
    16. Carceles-Poveda, Eva & Giannitsarou, Chryssi, 2007. "Adaptive learning in practice," Journal of Economic Dynamics and Control, Elsevier, vol. 31(8), pages 2659-2697, August.
    17. Detken, Carsten & Henry, Jérôme & Smets, Frank & Dieppe, Alistair & Marin, Carmen, 2002. "Model uncertainty and the equilibrium value of the real effective euro exchange rate," Working Paper Series 160, European Central Bank.
    18. Margaret M. McConnell & Gabriel Perez-Quiros, 2000. "Output fluctuations in the United States: what has changed since the early 1980s?," Proceedings, Federal Reserve Bank of San Francisco, issue Mar.
    19. Peter McAdam & Alpo Willman, 2004. "Supply, Factor Shares and Inflation Persistence: Re‐examining Euro‐area New‐Keynesian Phillips Curves," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 66(s1), pages 637-670, September.
    20. Stephen K. McNees, 1990. "Man vs. model? The role of judgment in forecasting," New England Economic Review, Federal Reserve Bank of Boston, issue Jul, pages 41-52.
    21. Fagan, Gabriel & Henry, Jerome & Mestre, Ricardo, 2005. "An area-wide model for the euro area," Economic Modelling, Elsevier, vol. 22(1), pages 39-59, January.
    22. Frank Smets & Raf Wouters, 2003. "An Estimated Dynamic Stochastic General Equilibrium Model of the Euro Area," Journal of the European Economic Association, MIT Press, vol. 1(5), pages 1123-1175, September.
    23. Clements, Michael P & Hendry, David F, 1996. "Intercept Corrections and Structural Change," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 11(5), pages 475-494, Sept.-Oct.
    24. Clements, Michael P. & Hendry, David F., 1998. "Forecasting economic processes," International Journal of Forecasting, Elsevier, vol. 14(1), pages 111-131, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Henzel, Steffen R. & Mayr, Johannes, 2013. "The mechanics of VAR forecast pooling—A DSGE model based Monte Carlo study," The North American Journal of Economics and Finance, Elsevier, vol. 24(C), pages 1-24.
    2. Kevin Clinton & Marianne Johnson & Mr. Jaromir Benes & Mr. Douglas Laxton & Mr. Troy D Matheson, 2010. "Structural Models in Real Time," IMF Working Papers 2010/056, International Monetary Fund.
    3. repec:onb:oenbwp:y::i:151:b:1 is not listed on IDEAS
    4. Lorena Skufi & Adam Geršl, 2023. "Using Macrofinancial Models to Simulate Macroeconomic Developments During the COVID-19 Pandemic: The Case of Albania," Eastern European Economics, Taylor & Francis Journals, vol. 61(5), pages 517-553, September.
    5. Georgios Papadopoulos & Dionysios Chionis & Nikolaos P. Rachaniotis, 2018. "Macro-financial linkages during tranquil and crisis periods: evidence from stressed economies," Risk Management, Palgrave Macmillan, vol. 20(2), pages 142-166, May.
    6. Christian Ragacs & Martin Schneider, 2009. "Why did we fail to predict GDP during the last cycle? A breakdown of forecast errors for Austria," Working Papers 151, Oesterreichische Nationalbank (Austrian Central Bank).
    7. Mahmut Gunay, 2018. "Nowcasting Annual Turkish GDP Growth with MIDAS," CBT Research Notes in Economics 1810, Research and Monetary Policy Department, Central Bank of the Republic of Turkey.
    8. Hjelm, Göran & Jönsson, Kristian, 2010. "In Search of a Method for Measuring the Output Gap of the Swedish Economy," Working Papers 115, National Institute of Economic Research.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. McAdam, Peter & Mestre, Ricardo, 2008. "Evaluating macro-economic models in the frequency domain: A note," Economic Modelling, Elsevier, vol. 25(6), pages 1137-1143, November.
    2. Anderson, Heather M. & Dungey, Mardi & Osborn, Denise R. & Vahid, Farshid, 2011. "Financial integration and the construction of historical financial data for the Euro Area," Economic Modelling, Elsevier, vol. 28(4), pages 1498-1509, July.
    3. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    4. ALISTAIR DIEPPE & KEITH KÜSTER & PETER McADAM, 2005. "Optimal Monetary Policy Rules for the Euro Area: An Analysis Using the Area Wide Model," Journal of Common Market Studies, Wiley Blackwell, vol. 43(3), pages 507-537, September.
    5. Gregory de Walque & Frank Smets & Rafael Wouters, 2006. "Firm-Specific Production Factors in a DSGE Model with Taylor Price Setting," International Journal of Central Banking, International Journal of Central Banking, vol. 2(3), September.
    6. Rebeca Jimenez-Rodriguez & Amalia Morales-Zumaquero & Balazs Egert, 2010. "The VARying Effect of Foreign Shocks in Central and Eastern Europe," William Davidson Institute Working Papers Series wp989, William Davidson Institute at the University of Michigan.
    7. Annicchiarico, Barbara & Giammarioli, Nicola & Piergallini, Alessandro, 2012. "Budgetary policies in a DSGE model with finite horizons," Research in Economics, Elsevier, vol. 66(2), pages 111-130.
    8. Keith Kuester & Volker Wieland, 2010. "Insurance Policies for Monetary Policy in the Euro Area," Journal of the European Economic Association, MIT Press, vol. 8(4), pages 872-912, June.
    9. Wieland, Volker & Cwik, Tobias & Müller, Gernot J. & Schmidt, Sebastian & Wolters, Maik, 2012. "A new comparative approach to macroeconomic modeling and policy analysis," Journal of Economic Behavior & Organization, Elsevier, vol. 83(3), pages 523-541.
    10. Eilev S. Jansen, 2004. "Modelling inflation in the Euro Area," Working Paper 2004/10, Norges Bank.
    11. Patrick Feve & Julien Matheron & Guillaume Sahuc, 2010. "Désinflation et chômage dans la zone euro : une analyse à l'aide d'un modèle var structurel," Annals of Economics and Statistics, GENES, issue 99-100, pages 365-394.
    12. Patrick F»Ve & Julien Matheron & Jean-Guillaume Sahuc, 2010. "Disinflation Shocks in the Eurozone: A DSGE Perspective," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 42(2-3), pages 289-323, March.
    13. Todd E. Clark & Michael W. McCracken, 2006. "Forecasting of small macroeconomic VARs in the presence of instabilities," Research Working Paper RWP 06-09, Federal Reserve Bank of Kansas City.
    14. Graham Elliott & Allan Timmermann, 2016. "Forecasting in Economics and Finance," Annual Review of Economics, Annual Reviews, vol. 8(1), pages 81-110, October.
    15. Levine, Paul & McAdam, Peter & Pearlman, Joseph, 2012. "Probability models and robust policy rules," European Economic Review, Elsevier, vol. 56(2), pages 246-262.
    16. McAdam, Peter & McNelis, Paul, 2005. "Forecasting inflation with thick models and neural networks," Economic Modelling, Elsevier, vol. 22(5), pages 848-867, September.
    17. Jérôme Coffinet & Céline Poilly, 2009. "Une évaluation structurelle du ratio de sacrifice dans la zone euro," Revue d'économie politique, Dalloz, vol. 119(2), pages 273-299.
    18. Beyer, Andreas & Juselius, Katarina, 2010. "Does it matter how aggregates are measured? The case of monetary transmission mechanisms in the euro area," Working Paper Series 1149, European Central Bank.
    19. Coenen, Gunter, 2007. "Inflation persistence and robust monetary policy design," Journal of Economic Dynamics and Control, Elsevier, vol. 31(1), pages 111-140, January.
    20. Clark, Todd E. & McCracken, Michael W., 2005. "The power of tests of predictive ability in the presence of structural breaks," Journal of Econometrics, Elsevier, vol. 124(1), pages 1-31, January.

    More about this item

    Keywords

    forecast accuracy; forecast projections; in-sample; macro-model; out-of-sample; structural break.;
    All these keywords.

    JEL classification:

    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • E30 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - General (includes Measurement and Data)
    • E32 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Business Fluctuations; Cycles
    • E37 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Forecasting and Simulation: Models and Applications

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ecb:ecbwps:2008950. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Official Publications (email available below). General contact details of provider: https://edirc.repec.org/data/emieude.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.