IDEAS home Printed from https://ideas.repec.org/a/spr/empeco/v64y2023i2d10.1007_s00181-022-02267-9.html
   My bibliography  Save this article

Will the last be the first? Ranking German macroeconomic forecasters based on different criteria

Author

Listed:
  • Tim Köhler

    (University of Applied Sciences Merseburg)

  • Jörg Döpke

    (University of Applied Sciences)

Abstract

We rank the quality of German macroeconomic forecasts using various methods for 17 regular annual German economic forecasts from 14 different institutions for the period from 1993 to 2019. Using data for just one year, rankings based on different methods correlate only weakly with each other. Correlations of rankings calculated for two consecutive years and a given method are often relatively low and statistically insignificant. For the total sample, rank correlations between institutions are generally relatively high among different criteria. We report substantial long-run differences in forecasting quality, which are mostly due to distinct average forecast horizons. In the long-run, choosing the criterion to rank the forecasters is of minor importance. Rankings based on recession years and normal periods are similar. The same does hold for rankings based on real-time vs revised data.

Suggested Citation

  • Tim Köhler & Jörg Döpke, 2023. "Will the last be the first? Ranking German macroeconomic forecasters based on different criteria," Empirical Economics, Springer, vol. 64(2), pages 797-832, February.
    • Handle: RePEc:spr:empeco:v:64:y:2023:i:2:d:10.1007_s00181-022-02267-9
    DOI: 10.1007/s00181-022-02267-9
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00181-022-02267-9
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00181-022-02267-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Clements, Michael P., 2019. "Do forecasters target first or later releases of national accounts data?," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1240-1249.
    2. Francis X. Diebold, 2015. "Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold-Mariano Tests," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 33(1), pages 1-1, January.
    3. Hyndman, Rob J. & Khandakar, Yeasmin, 2008. "Automatic Time Series Forecasting: The forecast Package for R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 27(i03).
    4. Tara M. Sinclair & H. O. Stekler & Warren Carnow, 2012. "A new approach for evaluating economic forecasts," Economics Bulletin, AccessEcon, vol. 32(3), pages 2332-2342.
    5. Antonello D’Agostino & Kieran Mcquinn & Karl Whelan, 2012. "Are Some Forecasters Really Better Than Others?," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 44(4), pages 715-732, June.
    6. Ullrich Heilemann & Herman O. Stekler, 2013. "Has The Accuracy of Macroeconomic Forecasts for Germany Improved?," German Economic Review, Verein für Socialpolitik, vol. 14(2), pages 235-253, May.
    7. Constantin Bürgi & Tara M. Sinclair, 2017. "A nonparametric approach to identifying a subset of forecasters that outperforms the simple average," Empirical Economics, Springer, vol. 53(1), pages 101-115, August.
    8. Francis X. Diebold & Jose A. Lopez, 1995. "Forecast evaluation and combination," Research Paper 9525, Federal Reserve Bank of New York.
    9. Casey, Eddie, 2020. "Do macroeconomic forecasters use macroeconomics to forecast?," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1439-1453.
    10. Rybinski, Krzysztof, 2021. "Ranking professional forecasters by the predictive power of their narratives," International Journal of Forecasting, Elsevier, vol. 37(1), pages 186-204.
    11. Döhrn, Roland, 2015. "Der Prognostiker des Jahres: Ein Zufallsergebnis? Möglichkeiten einer mehrdimensionalen Evaluierung von Konjunkturprognosen," IBES Diskussionsbeiträge 208, University of Duisburg-Essen, Institute of Business and Economic Studie (IBES).
    12. Timmermann, Allan & Qu, Ritong & Zhu, Yinchu, 2019. "Do Any Economists Have Superior Forecasting Skills?," CEPR Discussion Papers 14112, C.E.P.R. Discussion Papers.
    13. Gamber, Edward N. & Smith, Julie K. & McNamara, Dylan C., 2014. "Where is the Fed in the distribution of forecasters?," Journal of Policy Modeling, Elsevier, vol. 36(2), pages 296-312.
    14. Hyndman, Rob J. & Koehler, Anne B., 2006. "Another look at measures of forecast accuracy," International Journal of Forecasting, Elsevier, vol. 22(4), pages 679-688.
    15. Ullrich Heilemann & Karsten Müller, 2018. "Wenig Unterschiede – Zur Treffsicherheit Internationaler Prognosen und Prognostiker [Few differences—on the accuracy of international forecasts and forecaster]," AStA Wirtschafts- und Sozialstatistisches Archiv, Springer;Deutsche Statistische Gesellschaft - German Statistical Society, vol. 12(3), pages 195-233, December.
    16. Dovern, Jonas & Jannsen, Nils, 2017. "Systematic errors in growth expectations over the business cycle," International Journal of Forecasting, Elsevier, vol. 33(4), pages 760-769.
    17. Batchelor, R A, 1990. "All Forecasters Are Equal," Journal of Business & Economic Statistics, American Statistical Association, vol. 8(1), pages 143-144, January.
    18. Timmermann, Allan & Zhu, Yinchu, 2019. "Comparing Forecasting Performance with Panel Data," CEPR Discussion Papers 13746, C.E.P.R. Discussion Papers.
    19. Knüppel, Malte & Vladu, Andreea L., 2016. "Approximating fixed-horizon forecasts using fixed-event forecasts," Discussion Papers 28/2016, Deutsche Bundesbank.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Karsten Müller, 2022. "German forecasters’ narratives: How informative are German business cycle forecast reports?," Empirical Economics, Springer, vol. 62(5), pages 2373-2415, May.
    2. Döpke, Jörg & Müller, Karsten & Tegtmeier, Lars, 2018. "The economic value of business cycle forecasts for potential investors – Evidence from Germany," Research in International Business and Finance, Elsevier, vol. 46(C), pages 445-461.
    3. Döpke, Jörg & Fritsche, Ulrich & Müller, Karsten, 2019. "Has macroeconomic forecasting changed after the Great Recession? Panel-based evidence on forecast accuracy and forecaster behavior from Germany," Journal of Macroeconomics, Elsevier, vol. 62(C).
    4. Makridakis, Spyros & Hyndman, Rob J. & Petropoulos, Fotios, 2020. "Forecasting in social settings: The state of the art," International Journal of Forecasting, Elsevier, vol. 36(1), pages 15-28.
    5. Constantin Rudolf Salomo Bürgi, 2023. "How to deal with missing observations in surveys of professional forecasters," Journal of Applied Economics, Taylor & Francis Journals, vol. 26(1), pages 2185975-218, December.
    6. Theodosiou, Marina, 2011. "Forecasting monthly and quarterly time series using STL decomposition," International Journal of Forecasting, Elsevier, vol. 27(4), pages 1178-1195, October.
    7. Strunz, Franziska & Gödl, Maximilian, 2023. "An Evaluation of Professional Forecasts for the German Economy," VfS Annual Conference 2023 (Regensburg): Growth and the "sociale Frage" 277707, Verein für Socialpolitik / German Economic Association.
    8. Constantin Bürgi & Tara M. Sinclair, 2017. "A nonparametric approach to identifying a subset of forecasters that outperforms the simple average," Empirical Economics, Springer, vol. 53(1), pages 101-115, August.
    9. Frank J. Fabozzi & Francesco A. Fabozzi & Diana Tunaru, 2023. "A comparison of multi-factor term structure models for interbank rates," Review of Quantitative Finance and Accounting, Springer, vol. 61(1), pages 323-356, July.
    10. Nathan Goldstein & Ben‐Zion Zilberfarb, 2023. "The closer we get, the better we are?," Economic Inquiry, Western Economic Association International, vol. 61(2), pages 364-376, April.
    11. Jörg Döpke & Ulrich Fritsche & Karsten Müller, 2018. "Has Macroeconomic Forecasting changed after the Great Recession? - Panel-based Evidence on Accuracy and Forecaster Behaviour from Germany," Macroeconomics and Finance Series 201803, University of Hamburg, Department of Socioeconomics.
    12. Hewamalage, Hansika & Bergmeir, Christoph & Bandara, Kasun, 2021. "Recurrent Neural Networks for Time Series Forecasting: Current status and future directions," International Journal of Forecasting, Elsevier, vol. 37(1), pages 388-427.
    13. Kourentzes, Nikolaos & Petropoulos, Fotios & Trapero, Juan R., 2014. "Improving forecasting by estimating time series structural components across multiple frequencies," International Journal of Forecasting, Elsevier, vol. 30(2), pages 291-302.
    14. Alysha M De Livera, 2010. "Automatic forecasting with a modified exponential smoothing state space framework," Monash Econometrics and Business Statistics Working Papers 10/10, Monash University, Department of Econometrics and Business Statistics.
    15. Ulrich Heilemann & Susanne Schnorr-Bäcker, 2016. "Could The Start Of The German Recession 2008-2009 Have Been Foreseen? Evidence From Real-Time Data," Working Papers 2016-003, The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting.
    16. Hyndman, Rob J. & Khandakar, Yeasmin, 2008. "Automatic Time Series Forecasting: The forecast Package for R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 27(i03).
    17. Meira, Erick & Cyrino Oliveira, Fernando Luiz & de Menezes, Lilian M., 2022. "Forecasting natural gas consumption using Bagging and modified regularization techniques," Energy Economics, Elsevier, vol. 106(C).
    18. repec:lan:wpaper:470 is not listed on IDEAS
    19. Kang, Yanfei & Hyndman, Rob J. & Smith-Miles, Kate, 2017. "Visualising forecasting algorithm performance using time series instance spaces," International Journal of Forecasting, Elsevier, vol. 33(2), pages 345-358.
    20. Huber, Jakob & Stuckenschmidt, Heiner, 2020. "Daily retail demand forecasting using machine learning with emphasis on calendric special days," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1420-1438.
    21. Bergmeir, Christoph & Hyndman, Rob J. & Benítez, José M., 2016. "Bagging exponential smoothing methods using STL decomposition and Box–Cox transformation," International Journal of Forecasting, Elsevier, vol. 32(2), pages 303-312.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:empeco:v:64:y:2023:i:2:d:10.1007_s00181-022-02267-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.