IDEAS home Printed from https://ideas.repec.org/p/een/camaaa/2014-55.html
   My bibliography  Save this paper

Evaluating Forecasts of a Vector of Variables: a German Forecasting Competition

Author

Listed:
  • Hans Christian Müller-Dröge
  • Tara M. Sinclair
  • H.O. Stekler

Abstract

In this paper we present an evaluation of forecasts of a vector of variables of the German economy made by different institutions. Our method permits one to evaluate the forecasts for each year and then if one is interested to combine the years. We use our method to determine an overall winner for a forecasting competition across twenty-five different institutions for a single time period using a vector of eight key economic variables. Typically forecasting competitions are judged on a variable-by-variable basis, but our methodology allows us to determine how each competitor performed overall. We find that the Bundesbank was the overall winner for 2013.

Suggested Citation

  • Hans Christian Müller-Dröge & Tara M. Sinclair & H.O. Stekler, 2014. "Evaluating Forecasts of a Vector of Variables: a German Forecasting Competition," CAMA Working Papers 2014-55, Centre for Applied Macroeconomic Analysis, Crawford School of Public Policy, The Australian National University.
    • Handle: RePEc:een:camaaa:2014-55
    as

    Download full text from publisher

    File URL: https://cama.crawford.anu.edu.au/sites/default/files/publication/cama_crawford_anu_edu_au/2014-07/55_2014_muller-droge_sinclair_stekleer.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Ivana Komunjer & Michael T. Owyang, 2012. "Multivariate Forecast Evaluation and Rationality Testing," The Review of Economics and Statistics, MIT Press, vol. 94(4), pages 1066-1080, November.
    2. Ullrich Heilemann & Herman O. Stekler, 2013. "Has The Accuracy of Macroeconomic Forecasts for Germany Improved?," German Economic Review, Verein für Socialpolitik, vol. 14(2), pages 235-253, May.
    3. Fred Joutz & Michael P. Clements & Herman O. Stekler, 2007. "An evaluation of the forecasts of the federal reserve: a pooled approach," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(1), pages 121-136.
    4. Jordà, Òscar & Knüppel, Malte & Marcellino, Massimiliano, 2013. "Empirical simultaneous prediction regions for path-forecasts," International Journal of Forecasting, Elsevier, vol. 29(3), pages 456-468.
    5. Groen, Jan J.J. & Kapetanios, George & Price, Simon, 2009. "A real time evaluation of Bank of England forecasts of inflation and growth," International Journal of Forecasting, Elsevier, vol. 25(1), pages 74-80.
    6. Kajal Lahiri & Antony Davies & Xuguang Sheng, 2010. "Analyzing Three-Dimensional Panel Data of Forecasts," Discussion Papers 10-07, University at Albany, SUNY, Department of Economics.
    7. Holden, K & Peel, D A, 1990. "On Testing for Unbiasedness and Efficiency of Forecasts," The Manchester School of Economic & Social Studies, University of Manchester, vol. 58(2), pages 120-127, June.
    8. Robert A. Eisenbeis & Daniel F. Waggoner & Tao Zha, 2002. "Evaluating Wall Street Journal survey forecasters: a multivariate approach," FRB Atlanta Working Paper 2002-8, Federal Reserve Bank of Atlanta.
    9. Jörg Döpke & Ulrich Fritsche & Boriss Siliverstovs, 2010. "Evaluating German business cycle forecasts under an asymmetric loss function," OECD Journal: Journal of Business Cycle Measurement and Analysis, OECD Publishing, Centre for International Research on Economic Tendency Surveys, vol. 2010(1), pages 1-18.
    10. Ulrich K. Müller & Gebhard Kirchgässner, 2006. "Are forecasters reluctant to revise their predictions? Some German evidence," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 25(6), pages 401-413.
    11. Jacob A. Mincer & Victor Zarnowitz, 1969. "The Evaluation of Economic Forecasts," NBER Chapters, in: Economic Forecasts and Expectations: Analysis of Forecasting Behavior and Performance, pages 3-46, National Bureau of Economic Research, Inc.
    12. Andrew Bauer & Robert A. Eisenbeis & Daniel F. Waggoner & Tao Zha, 2003. "Forecast evaluation with cross-sectional data: The Blue Chip Surveys," Economic Review, Federal Reserve Bank of Atlanta, vol. 88(Q2), pages 17-31.
    13. Sinclair, Tara M. & Stekler, H.O., 2013. "Examining the quality of early GDP component estimates," International Journal of Forecasting, Elsevier, vol. 29(4), pages 736-750.
    14. Hsiao,Cheng & Pesaran,M. Hashem & Lahiri,Kajal & Lee,Lung Fei (ed.), 1999. "Analysis of Panels and Limited Dependent Variable Models," Cambridge Books, Cambridge University Press, number 9780521631693.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Dovern, Jonas & Feldkircher, Martin & Huber, Florian, 2016. "Does joint modelling of the world economy pay off? Evaluating global forecasts from a Bayesian GVAR," Journal of Economic Dynamics and Control, Elsevier, vol. 70(C), pages 86-100.
    2. Inske Pirschel & Maik H. Wolters, 2018. "Forecasting with large datasets: compressing information before, during or after the estimation?," Empirical Economics, Springer, vol. 55(2), pages 573-596, September.
    3. Ulrich Fritsche & Artur Tarassow, 2017. "Vergleichende Evaluation der Konjunkturprognosen des Instituts für Makroökonomie und Konjunkturforschung an der Hans-Böckler-Stiftung für den Zeitraum 2005-2014," IMK Studies 54-2017, IMK at the Hans Boeckler Foundation, Macroeconomic Policy Institute.
    4. Mihaela SIMIONESCU, 2015. "The Evaluation of Global Accuracy of Romanian Inflation Rate Predictions Using Mahalanobis Distance," Management Dynamics in the Knowledge Economy, College of Management, National University of Political Studies and Public Administration, vol. 3(1), pages 133-149, March.
    5. Dovern, Jonas, 2015. "A multivariate analysis of forecast disagreement: Confronting models of disagreement with survey data," European Economic Review, Elsevier, vol. 80(C), pages 16-35.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sinclair, Tara M. & Stekler, H.O. & Carnow, Warren, 2015. "Evaluating a vector of the Fed’s forecasts," International Journal of Forecasting, Elsevier, vol. 31(1), pages 157-164.
    2. Olga Isengildina‐Massa & Berna Karali & Todd H. Kuethe & Ani L. Katchova, 2021. "Joint Evaluation of the System of USDA's Farm Income Forecasts," Applied Economic Perspectives and Policy, John Wiley & Sons, vol. 43(3), pages 1140-1160, September.
    3. Tara M. Sinclair & H. O. Stekler & Warren Carnow, 2012. "A new approach for evaluating economic forecasts," Economics Bulletin, AccessEcon, vol. 32(3), pages 2332-2342.
    4. Tara M. Sinclair & H.O. Stekler, 2011. "Differences in Early GDP Component Estimates Between Recession and Expansion," Working Papers 2011-05, The George Washington University, Institute for International Economic Policy.
    5. Sinclair, Tara M. & Stekler, H.O., 2013. "Examining the quality of early GDP component estimates," International Journal of Forecasting, Elsevier, vol. 29(4), pages 736-750.
    6. Kappler, Marcus, 2007. "Projecting the Medium-Term: Outcomes and Errors for GDP Growth," ZEW Discussion Papers 07-068, ZEW - Leibniz Centre for European Economic Research.
    7. Behrens, Christoph, 2019. "Evaluating the Joint Efficiency of German Trade Forecasts. A nonparametric multivariate approach," Working Papers 9, German Research Foundation's Priority Programme 1859 "Experience and Expectation. Historical Foundations of Economic Behaviour", Humboldt University Berlin.
    8. Döpke, Jörg & Fritsche, Ulrich & Müller, Karsten, 2019. "Has macroeconomic forecasting changed after the Great Recession? Panel-based evidence on forecast accuracy and forecaster behavior from Germany," Journal of Macroeconomics, Elsevier, vol. 62(C).
    9. Fred Joutz & Michael P. Clements & Herman O. Stekler, 2007. "An evaluation of the forecasts of the federal reserve: a pooled approach," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(1), pages 121-136.
    10. Behrens, Christoph, 2020. "German trade forecasts since 1970: An evaluation using the panel dimension," Working Papers 26, German Research Foundation's Priority Programme 1859 "Experience and Expectation. Historical Foundations of Economic Behaviour", Humboldt University Berlin.
    11. Dovern, Jonas & Feldkircher, Martin & Huber, Florian, 2016. "Does joint modelling of the world economy pay off? Evaluating global forecasts from a Bayesian GVAR," Journal of Economic Dynamics and Control, Elsevier, vol. 70(C), pages 86-100.
    12. Ager, P. & Kappler, M. & Osterloh, S., 2009. "The accuracy and efficiency of the Consensus Forecasts: A further application and extension of the pooled approach," International Journal of Forecasting, Elsevier, vol. 25(1), pages 167-181.
    13. Constantin Burgi, 2016. "What Do We Lose When We Average Expectations?," Working Papers 2016-013, The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting.
    14. Jones, Jacob T. & Sinclair, Tara M. & Stekler, Herman O., 2020. "A textual analysis of Bank of England growth forecasts," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1478-1487.
    15. Sinclair, Tara M. & Gamber, Edward N. & Stekler, Herman & Reid, Elizabeth, 2012. "Jointly evaluating the Federal Reserve’s forecasts of GDP growth and inflation," International Journal of Forecasting, Elsevier, vol. 28(2), pages 309-314.
    16. Messina, Jeffrey D. & Sinclair, Tara M. & Stekler, Herman, 2015. "What can we learn from revisions to the Greenbook forecasts?," Journal of Macroeconomics, Elsevier, vol. 45(C), pages 54-62.
    17. Emmanuel C. Mamatzakis & Mike G. Tsionas, 2020. "Revealing forecaster's preferences: A Bayesian multivariate loss function approach," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 39(3), pages 412-437, April.
    18. Ines Fortin & Sebastian P. Koch & Klaus Weyerstrass, 2020. "Evaluation of economic forecasts for Austria," Empirical Economics, Springer, vol. 58(1), pages 107-137, January.
    19. Jörg Döpke & Ulrich Fritsche & Karsten Müller, 2018. "Has Macroeconomic Forecasting changed after the Great Recession? - Panel-based Evidence on Accuracy and Forecaster Behaviour from Germany," Macroeconomics and Finance Series 201803, University of Hamburg, Department of Socioeconomics.
    20. Alexander Foltas & Christian Pierdzioch, 2022. "Business-cycle reports and the efficiency of macroeconomic forecasts for Germany," Applied Economics Letters, Taylor & Francis Journals, vol. 29(10), pages 867-872, June.

    More about this item

    Keywords

    Mahalanobis Distance; forecasting competition; GDP components; German macroeconomic data;
    All these keywords.

    JEL classification:

    • C5 - Mathematical and Quantitative Methods - - Econometric Modeling
    • E2 - Macroeconomics and Monetary Economics - - Consumption, Saving, Production, Employment, and Investment
    • E3 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:een:camaaa:2014-55. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Cama Admin (email available below). General contact details of provider: https://edirc.repec.org/data/asanuau.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.