IDEAS home Printed from https://ideas.repec.org/a/wly/jforec/v41y2022i2p213-229.html
   My bibliography  Save this article

Evaluating the Eurosystem/ECB staff macroeconomic projections: The first 20 years

Author

Listed:
  • G. Kontogeorgos
  • K. Lambrias

Abstract

This work marks a thorough analysis of a confidential real‐time dataset consisting of the Eurosystem/ECB staff macroeconomic projections since their existence. By applying techniques widely employed in the literature of forecast evaluation, we examine their statistical properties with a special emphasis on optimality and rationality. Long‐term GDP projections are biased (tendency to overpredict), do not fully account for available information, and are outperformed by private sector expectations. Inflation projections are optimal and rational on a full‐sample analysis; however, subsample analysis reveals two distinct periods with a persistent and significant bias. Before the financial crisis inflation was persistently underpredicted, while in post‐2013, the bias reverses into overprediction.

Suggested Citation

  • G. Kontogeorgos & K. Lambrias, 2022. "Evaluating the Eurosystem/ECB staff macroeconomic projections: The first 20 years," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(2), pages 213-229, March.
    • Handle: RePEc:wly:jforec:v:41:y:2022:i:2:p:213-229
    DOI: 10.1002/for.2804
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/for.2804
    Download Restriction: no
    File URL: https://libkey.io/10.1002/for.2804?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jacob A. Mincer & Victor Zarnowitz, 1969. "The Evaluation of Economic Forecasts," NBER Chapters, in: Economic Forecasts and Expectations: Analysis of Forecasting Behavior and Performance, pages 3-46, National Bureau of Economic Research, Inc.
    2. Julien Champagne & Guillaume Poulin‐Bellisle & Rodrigo Sekkel, 2020. "Introducing the Bank of Canada staff economic projections database," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(1), pages 114-129, January.
    3. Lobato, Ignacio N. & Velasco, Carlos, 2004. "A Simple Test Of Normality For Time Series," Econometric Theory, Cambridge University Press, vol. 20(4), pages 671-689, August.
    4. Barbara Rossi & Tatevik Sekhposyan, 2016. "Forecast Rationality Tests in the Presence of Instabilities, with Applications to Federal Reserve and Survey Forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(3), pages 507-532, April.
    5. Whitney K. Newey & Kenneth D. West, 1994. "Automatic Lag Selection in Covariance Matrix Estimation," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 61(4), pages 631-653.
    6. Andrew Patton & Allan Timmermann, 2012. "Forecast Rationality Tests Based on Multi-Horizon Bounds," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(1), pages 1-17.
    7. El-Shagi, Makram & Giesen, Sebastian & Jung, Alexander, 2016. "Revisiting the relative forecast performances of Fed staff and private forecasters: A dynamic approach," International Journal of Forecasting, Elsevier, vol. 32(2), pages 313-323.
    8. Fildes, Robert & Stekler, Herman, 2002. "The state of macroeconomic forecasting," Journal of Macroeconomics, Elsevier, vol. 24(4), pages 435-468, December.
    9. Gavin, William T. & Mandal, Rachel J., 2003. "Evaluating FOMC forecasts," International Journal of Forecasting, Elsevier, vol. 19(4), pages 655-667.
    10. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    11. Graham Elliott & Allan Timmermann & Ivana Komunjer, 2005. "Estimation and Testing of Forecast Rationality under Flexible Loss," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 72(4), pages 1107-1125.
    12. Fildes, Robert & Stekler, Herman, 2002. "Reply to the comments on 'The state of macroeconomic forecasting'," Journal of Macroeconomics, Elsevier, vol. 24(4), pages 503-505, December.
    13. Cumby, Robert E & Huizinga, John, 1992. "Testing the Autocorrelation Structure of Disturbances in Ordinary Least Squares and Instrumental Variables Regressions," Econometrica, Econometric Society, vol. 60(1), pages 185-195, January.
    14. Andrews, Donald W K, 1991. "Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation," Econometrica, Econometric Society, vol. 59(3), pages 817-858, May.
    15. Graham Elliott & Allan Timmermann, 2016. "Economic Forecasting," Economics Books, Princeton University Press, edition 1, number 10740.
    16. Jushan Bai & Serena Ng, 2005. "Tests for Skewness, Kurtosis, and Normality for Time Series Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 49-60, January.
    17. Bobeica, Elena & Jarociński, Marek, 2017. "Missing disinflation and missing inflation," Research Bulletin, European Central Bank, vol. 30.
    18. Jacob A. Mincer, 1969. "Economic Forecasts and Expectations: Analysis of Forecasting Behavior and Performance," NBER Books, National Bureau of Economic Research, Inc, number minc69-1, March.
    19. Ciccarelli, Matteo & Osbat, Chiara, 2017. "Low inflation in the euro area: Causes and consequences," Occasional Paper Series 181, European Central Bank.
    20. Lucia Alessi & Eric Ghysels & Luca Onorante & Richard Peach & Simon Potter, 2014. "Central Bank Macroeconomic Forecasting During the Global Financial Crisis: The European Central Bank and Federal Reserve Bank of New York Experiences," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 32(4), pages 483-500, October.
    21. Patton, Andrew J. & Timmermann, Allan, 2007. "Testing Forecast Optimality Under Unknown Loss," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 1172-1184, December.
    22. Bobeica, Elena & Jarociński, Marek, 2017. "Missing disinflation and missing inflation," Research Bulletin, European Central Bank, vol. 30.
    23. Elena Bobeica & Marek Jarociński, 2019. "Missing Disinflation and Missing Inflation: A VAR Perspective," International Journal of Central Banking, International Journal of Central Banking, vol. 15(1), pages 199-232, March.
    24. Messina, Jeffrey D. & Sinclair, Tara M. & Stekler, Herman, 2015. "What can we learn from revisions to the Greenbook forecasts?," Journal of Macroeconomics, Elsevier, vol. 45(C), pages 54-62.
    25. Wolak, Frank A., 1989. "Testing inequality constraints in linear econometric models," Journal of Econometrics, Elsevier, vol. 41(2), pages 205-235, June.
    26. Philipp Hartman & Frank Smets, 2018. "The European Central Bank’s Monetary Policy during Its First 20 Years," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 49(2 (Fall)), pages 1-146.
    27. repec:zbw:bofrdp:2017_029 is not listed on IDEAS
    28. Graham Elliott & Allan Timmermann, 2016. "Forecasting in Economics and Finance," Annual Review of Economics, Annual Reviews, vol. 8(1), pages 81-110, October.
    29. David H. Romer & Christina D. Romer, 2000. "Federal Reserve Information and the Behavior of Interest Rates," American Economic Review, American Economic Association, vol. 90(3), pages 429-457, June.
    30. Peter Tulip & Stephanie Wallace, 2012. "Estimates of Uncertainty around the RBA's Forecasts," RBA Research Discussion Papers rdp2012-07, Reserve Bank of Australia.
    31. Holden, K & Peel, D A, 1990. "On Testing for Unbiasedness and Efficiency of Forecasts," The Manchester School of Economic & Social Studies, University of Manchester, vol. 58(2), pages 120-127, June.
    32. Pesaran, M. Hashem & Timmermann, Allan, 2009. "Testing Dependence Among Serially Correlated Multicategory Variables," Journal of the American Statistical Association, American Statistical Association, vol. 104(485), pages 325-337.
    33. Patton, Andrew J. & Timmermann, Allan, 2007. "Properties of optimal forecasts under asymmetric loss and nonlinearity," Journal of Econometrics, Elsevier, vol. 140(2), pages 884-918, October.
    34. Blaskowitz, Oliver & Herwartz, Helmut, 2014. "Testing the value of directional forecasts in the presence of serial correlation," International Journal of Forecasting, Elsevier, vol. 30(1), pages 30-42.
    35. Page, Adrian & Lambrias, Kyriacos, 2019. "The performance of the Eurosystem/ECB staff macroeconomic projections since the financial crisis," Economic Bulletin Articles, European Central Bank, vol. 8.
    36. Faust, Jon & Wright, Jonathan H., 2008. "Efficient forecast tests for conditional policy forecasts," Journal of Econometrics, Elsevier, vol. 146(2), pages 293-303, October.
    37. Issing, O., 2004. "The role of macroeconomic projections within the monetary policy strategy of the ECB," Economic Modelling, Elsevier, vol. 21(5), pages 723-734, September.
    38. Joutz, Fred & Stekler, H. O., 2000. "An evaluation of the predictions of the Federal Reserve," International Journal of Forecasting, Elsevier, vol. 16(1), pages 17-38.
    39. Harvey, David & Leybourne, Stephen & Newbold, Paul, 1997. "Testing the equality of prediction mean squared errors," International Journal of Forecasting, Elsevier, vol. 13(2), pages 281-291, June.
    40. Reifschneider, David & Tulip, Peter, 2019. "Gauging the uncertainty of the economic outlook using historical forecasting errors: The Federal Reserve’s approach," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1564-1582.
    41. Hartmann, Philipp & Smets, Frank, 2018. "The first twenty years of the European Central Bank: monetary policy," Working Paper Series 2219, European Central Bank.
    42. Rostagno, Massimo & Altavilla, Carlo & Carboni, Giacomo & Lemke, Wolfgang & Motto, Roberto & Saint Guilhem, Arthur & Yiangou, Jonathan, 2019. "A tale of two decades: the ECB’s monetary policy at 20," Working Paper Series 2346, European Central Bank.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Carola Conces Binder & Rodrigo Sekkel, 2023. "Central Bank Forecasting: A Survey," Staff Working Papers 23-18, Bank of Canada.
    2. Goodhart, C. A. E. & Pradhan, Manoj, 2023. "A snapshot of Central Bank (two year) forecasting: a mixed picture," LSE Research Online Documents on Economics 118680, London School of Economics and Political Science, LSE Library.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kontogeorgos, Georgios & Lambrias, Kyriacos, 2019. "An analysis of the Eurosystem/ECB projections," Working Paper Series 2291, European Central Bank.
    2. Carola Conces Binder & Rodrigo Sekkel, 2023. "Central Bank Forecasting: A Survey," Staff Working Papers 23-18, Bank of Canada.
    3. Xie, Zixiong & Hsu, Shih-Hsun, 2016. "Time varying biases and the state of the economy," International Journal of Forecasting, Elsevier, vol. 32(3), pages 716-725.
    4. Dovern, Jonas & Jannsen, Nils, 2017. "Systematic errors in growth expectations over the business cycle," International Journal of Forecasting, Elsevier, vol. 33(4), pages 760-769.
    5. Constantin Burgi, 2016. "What Do We Lose When We Average Expectations?," Working Papers 2016-013, The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting.
    6. Arai, Natsuki, 2020. "Investigating the inefficiency of the CBO’s budgetary projections," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1290-1300.
    7. Messina, Jeffrey D. & Sinclair, Tara M. & Stekler, Herman, 2015. "What can we learn from revisions to the Greenbook forecasts?," Journal of Macroeconomics, Elsevier, vol. 45(C), pages 54-62.
    8. Timo Dimitriadis & Andrew J. Patton & Patrick W. Schmidt, 2019. "Testing Forecast Rationality for Measures of Central Tendency," Papers 1910.12545, arXiv.org, revised Jun 2023.
    9. Ulrich Fritsche & Artur Tarassow, 2017. "Vergleichende Evaluation der Konjunkturprognosen des Instituts für Makroökonomie und Konjunkturforschung an der Hans-Böckler-Stiftung für den Zeitraum 2005-2014," IMK Studies 54-2017, IMK at the Hans Boeckler Foundation, Macroeconomic Policy Institute.
    10. Faust, Jon & Wright, Jonathan H., 2013. "Forecasting Inflation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 2-56, Elsevier.
    11. Granziera, Eleonora & Jalasjoki, Pirkka & Paloviita, Maritta, 2021. "The bias and efficiency of the ECB inflation projections: a State dependent analysis," Research Discussion Papers 7/2021, Bank of Finland.
    12. Andrew C. Chang & Trace J. Levinson, 2023. "Raiders of the lost high‐frequency forecasts: New data and evidence on the efficiency of the Fed's forecasting," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(1), pages 88-104, January.
    13. Pincheira, Pablo & Hardy, Nicolas, 2022. "Correlation Based Tests of Predictability," MPRA Paper 112014, University Library of Munich, Germany.
    14. Ericsson, Neil R., 2017. "How biased are U.S. government forecasts of the federal debt?," International Journal of Forecasting, Elsevier, vol. 33(2), pages 543-559.
    15. repec:zbw:bofrdp:2021_007 is not listed on IDEAS
    16. Robert P. Lieli & Augusto Nieto-Barthaburu, 2023. "Forecasting with Feedback," Papers 2308.15062, arXiv.org, revised Jan 2024.
    17. Eleonora Granziera & Pirkka Jalasjoki & Maritta Paloviita, 2021. "The Bias and Efficiency of the ECB Inflation Projections: a State Dependent Analysis," Working Paper 2021/1, Norges Bank.
    18. Wang, Yiyao & Lee, Tae-Hwy, 2014. "Asymmetric loss in the Greenbook and the Survey of Professional Forecasters," International Journal of Forecasting, Elsevier, vol. 30(2), pages 235-245.
    19. Ericsson, Neil R., 2016. "Eliciting GDP forecasts from the FOMC’s minutes around the financial crisis," International Journal of Forecasting, Elsevier, vol. 32(2), pages 571-583.
    20. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    21. Barbara Rossi, 2019. "Forecasting in the presence of instabilities: How do we know whether models predict well and how to improve them," Economics Working Papers 1711, Department of Economics and Business, Universitat Pompeu Fabra, revised Jul 2021.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:jforec:v:41:y:2022:i:2:p:213-229. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/2966 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.