IDEAS home Printed from https://ideas.repec.org/p/ecb/ecbwps/2008950.html

Is forecasting with large models informative? Assessing the role of judgement in macroeconomic forecasts

Author

Listed:
  • McAdam, Peter
  • Mestre, Ricardo

Abstract

We evaluate residual projection strategies in the context of a large-scale macro model of the euro area and smaller benchmark time-series models. The exercises attempt to measure the accuracy of model-based forecasts simulated both out-of-sample and in-sample. Both exercises incorporate alternative residual-projection methods, to assess the importance of unaccounted-for breaks in forecast accuracy and off-model judgment. Conclusions reached are that simple mechanical residual adjustments have a significant impact of forecasting accuracy irrespective of the model in use, ostensibly due to the presence of breaks in trends in the data. The testing procedure and conclusions are applicable to a wide class of models and thus of general interest. JEL Classification: C52, E30, E32, E37

Suggested Citation

  • McAdam, Peter & Mestre, Ricardo, 2008. "Is forecasting with large models informative? Assessing the role of judgement in macroeconomic forecasts," Working Paper Series 950, European Central Bank.
  • Handle: RePEc:ecb:ecbwps:2008950
    Note: 50336
    as

    Download full text from publisher

    File URL: https://www.ecb.europa.eu//pub/pdf/scpwps/ecbwp950.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Benati, Luca & Vitale, Giovanni, 2007. "Joint estimation of the natural rate of interest, the natural rate of unemployment, expected inflation, and potential output," Working Paper Series 797, European Central Bank.
    2. Christopher A. Sims, 2002. "The Role of Models and Probabilities in the Monetary Policy Process," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 33(2), pages 1-62.
    3. G. Coenen & J.-L. Vega, 2001. "The demand for M3 in the euro area," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 16(6), pages 727-748.
    4. Todd Clark & Michael McCracken, 2005. "Evaluating Direct Multistep Forecasts," Econometric Reviews, Taylor & Francis Journals, vol. 24(4), pages 369-404.
    5. John F. Henry & L. Randall Wray, 1998. "Economic Time," Macroeconomics 9811004, University Library of Munich, Germany.
    6. Gali, Jordi & Gertler, Mark & Lopez-Salido, J. David, 2001. "European inflation dynamics," European Economic Review, Elsevier, vol. 45(7), pages 1237-1270.
    7. Detken, Carsten & Henry, Jérôme & Smets, Frank & Dieppe, Alistair & Marin, Carmen, 2002. "Model uncertainty and the equilibrium value of the real effective euro exchange rate," Working Paper Series 160, European Central Bank.
    8. Gabriel Perez-Quiros & Margaret M. McConnell, 2000. "Output Fluctuations in the United States: What Has Changed since the Early 1980's?," American Economic Review, American Economic Association, vol. 90(5), pages 1464-1476, December.
    9. Marcellino, Massimiliano & Stock, James H. & Watson, Mark W., 2003. "Macroeconomic forecasting in the Euro area: Country specific versus area-wide information," European Economic Review, Elsevier, vol. 47(1), pages 1-18, February.
    10. Carceles-Poveda, Eva & Giannitsarou, Chryssi, 2007. "Adaptive learning in practice," Journal of Economic Dynamics and Control, Elsevier, vol. 31(8), pages 2659-2697, August.
    11. Atsushi Inoue & Lutz Kilian, 2005. "In-Sample or Out-of-Sample Tests of Predictability: Which One Should We Use?," Econometric Reviews, Taylor & Francis Journals, vol. 23(4), pages 371-402.
    12. Peter McAdam & Alpo Willman, 2004. "Supply, Factor Shares and Inflation Persistence: Re‐examining Euro‐area New‐Keynesian Phillips Curves," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 66(s1), pages 637-670, September.
    13. Ramón Adalid & Günter Coenen & Peter McAdam & Stefano Siviero, 2005. "The Performance and Robustness of Interest-Rate Rules in Models of the Euro Area," International Journal of Central Banking, International Journal of Central Banking, vol. 1(1), May.
    14. Stephen K. McNees, 1990. "Man vs. model? The role of judgment in forecasting," New England Economic Review, Federal Reserve Bank of Boston, issue Jul, pages 41-52.
    15. Stock, James H & Watson, Mark W, 1996. "Evidence on Structural Instability in Macroeconomic Time Series Relations," Journal of Business & Economic Statistics, American Statistical Association, vol. 14(1), pages 11-30, January.
    16. Stock, James H. & Watson, Mark W., 1999. "Business cycle fluctuations in us macroeconomic time series," Handbook of Macroeconomics, in: J. B. Taylor & M. Woodford (ed.), Handbook of Macroeconomics, edition 1, volume 1, chapter 1, pages 3-64, Elsevier.
    17. Jushan Bai & Pierre Perron, 2003. "Computation and analysis of multiple structural change models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 18(1), pages 1-22.
    18. Brock, William A. & Durlauf, Steven N. & West, Kenneth D., 2007. "Model uncertainty and policy evaluation: Some theory and empirics," Journal of Econometrics, Elsevier, vol. 136(2), pages 629-664, February.
    19. Stefano Siviero & Daniele Terlizzese, 2008. "Macroeconomic Forecasting: Debunking a Few Old Wives' Tales," Journal of Business Cycle Measurement and Analysis, OECD Publishing, Centre for International Research on Economic Tendency Surveys, vol. 2007(3), pages 287-316.
    20. Fagan, Gabriel & Henry, Jerome & Mestre, Ricardo, 2005. "An area-wide model for the euro area," Economic Modelling, Elsevier, vol. 22(1), pages 39-59, January.
    21. Frank Smets & Raf Wouters, 2003. "An Estimated Dynamic Stochastic General Equilibrium Model of the Euro Area," Journal of the European Economic Association, MIT Press, vol. 1(5), pages 1123-1175, September.
    22. Clements, Michael P & Hendry, David F, 1996. "Intercept Corrections and Structural Change," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 11(5), pages 475-494, Sept.-Oct.
    23. Dieppe, Alistair & McAdam, Peter, 2006. "Monetary policy under a liquidity trap: Simulation evidence for the euro area," Journal of the Japanese and International Economies, Elsevier, vol. 20(3), pages 338-363, September.
    24. Clements, Michael P. & Hendry, David F., 1998. "Forecasting economic processes," International Journal of Forecasting, Elsevier, vol. 14(1), pages 111-131, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Henzel, Steffen R. & Mayr, Johannes, 2013. "The mechanics of VAR forecast pooling—A DSGE model based Monte Carlo study," The North American Journal of Economics and Finance, Elsevier, vol. 24(C), pages 1-24.
    2. Kevin Clinton & Marianne Johnson & Mr. Jaromir Benes & Mr. Douglas Laxton & Mr. Troy D Matheson, 2010. "Structural Models in Real Time," IMF Working Papers 2010/056, International Monetary Fund.
    3. repec:onb:oenbwp:y::i:151:b:1 is not listed on IDEAS
    4. Lorena Skufi & Adam Geršl, 2023. "Using Macrofinancial Models to Simulate Macroeconomic Developments During the COVID-19 Pandemic: The Case of Albania," Eastern European Economics, Taylor & Francis Journals, vol. 61(5), pages 517-553, September.
    5. Georgios Papadopoulos & Dionysios Chionis & Nikolaos P. Rachaniotis, 2018. "Macro-financial linkages during tranquil and crisis periods: evidence from stressed economies," Risk Management, Palgrave Macmillan, vol. 20(2), pages 142-166, May.
    6. Christian Ragacs & Martin Schneider, 2009. "Why did we fail to predict GDP during the last cycle? A breakdown of forecast errors for Austria," Working Papers 151, Oesterreichische Nationalbank (Austrian Central Bank).
    7. Mahmut Gunay, 2018. "Nowcasting Annual Turkish GDP Growth with MIDAS," CBT Research Notes in Economics 1810, Research and Monetary Policy Department, Central Bank of the Republic of Turkey.
    8. Hjelm, Göran & Jönsson, Kristian, 2010. "In Search of a Method for Measuring the Output Gap of the Swedish Economy," Working Papers 115, National Institute of Economic Research.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. McAdam, Peter & Mestre, Ricardo, 2008. "Evaluating macro-economic models in the frequency domain: A note," Economic Modelling, Elsevier, vol. 25(6), pages 1137-1143, November.
    2. Anderson, Heather M. & Dungey, Mardi & Osborn, Denise R. & Vahid, Farshid, 2011. "Financial integration and the construction of historical financial data for the Euro Area," Economic Modelling, Elsevier, vol. 28(4), pages 1498-1509, July.
    3. Annicchiarico, Barbara & Giammarioli, Nicola & Piergallini, Alessandro, 2012. "Budgetary policies in a DSGE model with finite horizons," Research in Economics, Elsevier, vol. 66(2), pages 111-130.
    4. Wieland, Volker & Cwik, Tobias & Müller, Gernot J. & Schmidt, Sebastian & Wolters, Maik, 2012. "A new comparative approach to macroeconomic modeling and policy analysis," Journal of Economic Behavior & Organization, Elsevier, vol. 83(3), pages 523-541.
    5. Keith Kuester & Volker Wieland, 2010. "Insurance Policies for Monetary Policy in the Euro Area," Journal of the European Economic Association, MIT Press, vol. 8(4), pages 872-912, June.
    6. Levine, Paul & McAdam, Peter & Pearlman, Joseph, 2012. "Probability models and robust policy rules," European Economic Review, Elsevier, vol. 56(2), pages 246-262.
    7. Gregory de Walque & Frank Smets & Rafael Wouters, 2006. "Firm-Specific Production Factors in a DSGE Model with Taylor Price Setting," International Journal of Central Banking, International Journal of Central Banking, vol. 2(3), September.
    8. Rebeca Jimenez-Rodriguez & Amalia Morales-Zumaquero & Balazs Egert, 2010. "The VARying Effect of Foreign Shocks in Central and Eastern Europe," William Davidson Institute Working Papers Series wp989, William Davidson Institute at the University of Michigan.
    9. Todd E. Clark & Michael W. McCracken, 2006. "Forecasting of small macroeconomic VARs in the presence of instabilities," Research Working Paper RWP 06-09, Federal Reserve Bank of Kansas City.
    10. Graham Elliott & Allan Timmermann, 2016. "Forecasting in Economics and Finance," Annual Review of Economics, Annual Reviews, vol. 8(1), pages 81-110, October.
    11. Beyer, Andreas & Juselius, Katarina, 2010. "Does it matter how aggregates are measured? The case of monetary transmission mechanisms in the euro area," Working Paper Series 1149, European Central Bank.
    12. Jérôme Coffinet & Céline Poilly, 2009. "Une évaluation structurelle du ratio de sacrifice dans la zone euro," Revue d'économie politique, Dalloz, vol. 119(2), pages 273-299.
    13. Clark, Todd E. & McCracken, Michael W., 2005. "The power of tests of predictive ability in the presence of structural breaks," Journal of Econometrics, Elsevier, vol. 124(1), pages 1-31, January.
    14. Coenen, Gunter, 2007. "Inflation persistence and robust monetary policy design," Journal of Economic Dynamics and Control, Elsevier, vol. 31(1), pages 111-140, January.
    15. Kenneth S. Rogoff & Vania Stavrakeva, 2008. "The Continuing Puzzle of Short Horizon Exchange Rate Forecasting," NBER Working Papers 14071, National Bureau of Economic Research, Inc.
    16. ALISTAIR DIEPPE & KEITH KÜSTER & PETER McADAM, 2005. "Optimal Monetary Policy Rules for the Euro Area: An Analysis Using the Area Wide Model," Journal of Common Market Studies, Wiley Blackwell, vol. 43(3), pages 507-537, September.
    17. Marco Del Negro & Frank Schorfheide, 2009. "Monetary Policy Analysis with Potentially Misspecified Models," American Economic Review, American Economic Association, vol. 99(4), pages 1415-1450, September.
    18. Mohitosh Kejriwal, 2020. "A Robust Sequential Procedure for Estimating the Number of Structural Changes in Persistence," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 82(3), pages 669-685, June.
    19. Wieland, Volker & Wolters, Maik, 2013. "Forecasting and Policy Making," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 239-325, Elsevier.
    20. Smets, Frank & Warne, Anders & Wouters, Raf, 2013. "Professional forecasters and the real-time forecasting performance of an estimated new keynesian model for the euro area," Working Paper Series 1571, European Central Bank.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • E30 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - General (includes Measurement and Data)
    • E32 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Business Fluctuations; Cycles
    • E37 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Forecasting and Simulation: Models and Applications

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ecb:ecbwps:2008950. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Official Publications (email available below). General contact details of provider: https://edirc.repec.org/data/emieude.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.