IDEAS home Printed from https://ideas.repec.org/p/tky/fseres/2010cf729.html
   My bibliography  Save this paper

Evaluating Macroeconomic Forecasts: A Review of Some Recent Developments

Author

Listed:
  • Philip Hans Franses

    (Erasmus School of Economics, Erasmus University Rotterdam)

  • Michael McAleer

    (Erasmus School of Economics, Erasmus University Rotterdam and Tinbergen Institute)

  • Rianne Legerstee

    (Erasmus School of Economics, Erasmus University Rotterdam and Tinbergen Institute)

Abstract

Macroeconomic forecasts are frequently produced, published, discussed and used. The formal evaluation of such forecasts has a long research history. Recently, a new angle to the evaluation of forecasts has been addressed, and in this review we analyse some recent developments from that perspective. The literature on forecast evaluation predominantly assumes that macroeconomic forecasts are generated from econometric models. In practice, however, most macroeconomic forecasts, such as those from the IMF, World Bank, OECD, Federal Reserve Board, Federal Open Market Committee (FOMC) and the ECB, are based on econometric model forecasts as well as on human intuition. This seemingly inevitable combination renders most of these forecasts biased and, as such, their evaluation becomes non-standard. In this review, we consider the evaluation of two forecasts in which: (i) the two forecasts are generated from two distinct econometric models; (ii) one forecast is generated from an econometric model and the other is obtained as a combination of a model, the other forecast, and intuition; and (iii) the two forecasts are generated from two distinct combinations of different models and intuition. It is shown that alternative tools are needed to compare and evaluate the forecasts in each of these three situations. These alternative techniques are illustrated by comparing the forecasts from the Federal Reserve Board and the FOMC on inflation, unemployment and real GDP growth.

Suggested Citation

  • Philip Hans Franses & Michael McAleer & Rianne Legerstee, 2010. "Evaluating Macroeconomic Forecasts: A Review of Some Recent Developments," CIRJE F-Series CIRJE-F-729, CIRJE, Faculty of Economics, University of Tokyo.
  • Handle: RePEc:tky:fseres:2010cf729
    as

    Download full text from publisher

    File URL: http://www.cirje.e.u-tokyo.ac.jp/research/dp/2010/2010cf729.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Philip Hans Franses & Michael McAleer & Rianne Legerstee, 2009. "Expert opinion versus expertise in forecasting," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 63(3), pages 334-346, August.
    2. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    3. Batchelor, Roy, 2007. "Bias in macroeconomic forecasts," International Journal of Forecasting, Elsevier, vol. 23(2), pages 189-203.
    4. Graham Elliott & Allan Timmermann, 2016. "Economic Forecasting," Economics Books, Princeton University Press, edition 1, number 10740.
    5. Yock Y. Chong & David F. Hendry, 1986. "Econometric Evaluation of Linear Macro-Economic Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 53(4), pages 671-690.
    6. Timmermann, Allan, 2006. "Forecast Combinations," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 4, pages 135-196, Elsevier.
    7. Roy Batchelor, 2007. "Forecaster Behaviour and Bias in Macroeconomic Forecasts," ifo Working Paper Series 39, ifo Institute - Leibniz Institute for Economic Research at the University of Munich.
    8. Fiebig, Denzil G. & McAleer, Michael & Bartels, Robert, 1992. "Properties of ordinary least squares estimators in regression models with nonspherical disturbances," Journal of Econometrics, Elsevier, vol. 54(1-3), pages 321-334.
    9. Christina D. Romer & David H. Romer, 2004. "A New Measure of Monetary Shocks: Derivation and Implications," American Economic Review, American Economic Association, vol. 94(4), pages 1055-1084, September.
    10. Granger, C. W. J. & Newbold, Paul, 1986. "Forecasting Economic Time Series," Elsevier Monographs, Elsevier, edition 2, number 9780122951831 edited by Shell, Karl.
    11. G. Elliott & C. Granger & A. Timmermann (ed.), 2006. "Handbook of Economic Forecasting," Handbook of Economic Forecasting, Elsevier, edition 1, volume 1, number 1.
    12. Chang, Chia-Lin & Franses, Philip Hans & McAleer, Michael, 2011. "How accurate are government forecasts of economic fundamentals? The case of Taiwan," International Journal of Forecasting, Elsevier, vol. 27(4), pages 1066-1075, October.
    13. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
    14. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    15. Christina D. Romer & David H. Romer, 2008. "The FOMC versus the Staff: Where Can Monetary Policymakers Add Value?," American Economic Review, American Economic Association, vol. 98(2), pages 230-235, May.
    16. McAleer, Michael, 1992. "Efficient Estimation: The Rao-Zyskind Condition, Kruskal's Theorem and Ordinary Least Squares," The Economic Record, The Economic Society of Australia, vol. 68(200), pages 65-72, March.
    17. MICHAEL McALEER, 1992. "Efficient Estimation: The Rao‐Zyskind Condition, Kruskal's Theorem and Ordinary Least Squares," The Economic Record, The Economic Society of Australia, vol. 68(1), pages 65-72, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. repec:aaa:journl:v:3:y:1999:i:1:p:87-100 is not listed on IDEAS
    2. Constantin ANGHELACHE & Cristina SACALA, 2016. "Theoretical model used for macroeconomic analysis," Romanian Statistical Review Supplement, Romanian Statistical Review, vol. 64(7), pages 57-60, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Philip Hans Franses & Michael McAleer & Rianne Legerstee, 2014. "Evaluating Macroeconomic Forecasts: A Concise Review Of Some Recent Developments," Journal of Economic Surveys, Wiley Blackwell, vol. 28(2), pages 195-208, April.
    2. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    3. Clements, Michael P. & Harvey, David I., 2011. "Combining probability forecasts," International Journal of Forecasting, Elsevier, vol. 27(2), pages 208-223.
    4. Chang, Chia-Lin & Franses, Philip Hans & McAleer, Michael, 2013. "Are forecast updates progressive?," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 93(C), pages 9-18.
    5. Chang, Chia-Lin & Franses, Philip Hans & McAleer, Michael, 2011. "How accurate are government forecasts of economic fundamentals? The case of Taiwan," International Journal of Forecasting, Elsevier, vol. 27(4), pages 1066-1075, October.
    6. Raffaella Giacomini & Barbara Rossi, 2013. "Forecasting in macroeconomics," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 17, pages 381-408, Edward Elgar Publishing.
    7. Constantin Burgi, 2016. "What Do We Lose When We Average Expectations?," Working Papers 2016-013, The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting.
    8. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    9. Jamali, Ibrahim & Yamani, Ehab, 2019. "Out-of-sample exchange rate predictability in emerging markets: Fundamentals versus technical analysis," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 61(C), pages 241-263.
    10. Rapach, David E. & Strauss, Jack K., 2012. "Forecasting US state-level employment growth: An amalgamation approach," International Journal of Forecasting, Elsevier, vol. 28(2), pages 315-327.
    11. Lahiri, Kajal & Yang, Liu, 2013. "Forecasting Binary Outcomes," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1025-1106, Elsevier.
    12. Rossi, Barbara, 2006. "Are Exchange Rates Really Random Walks? Some Evidence Robust To Parameter Instability," Macroeconomic Dynamics, Cambridge University Press, vol. 10(1), pages 20-38, February.
    13. Zanetti Chini, Emilio, 2018. "Forecasting dynamically asymmetric fluctuations of the U.S. business cycle," International Journal of Forecasting, Elsevier, vol. 34(4), pages 711-732.
    14. Franses, Ph.H.B.F. & McAleer, M.J. & Legerstee, R., 2008. "Does the ROMC have expertise, and can it forecast?," Econometric Institute Research Papers EI 2008-33, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    15. Issler, João Victor & Lima, Luiz Renato, 2009. "A panel data approach to economic forecasting: The bias-corrected average forecast," Journal of Econometrics, Elsevier, vol. 152(2), pages 153-164, October.
    16. Barbara Rossi & Atsushi Inoue, 2012. "Out-of-Sample Forecast Tests Robust to the Choice of Window Size," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(3), pages 432-453, April.
    17. Guidolin, Massimo & Timmermann, Allan, 2009. "Forecasts of US short-term interest rates: A flexible forecast combination approach," Journal of Econometrics, Elsevier, vol. 150(2), pages 297-311, June.
    18. Clements, Michael P. & Galvao, Ana Beatriz, 2006. "Macroeconomic Forecasting with Mixed Frequency Data: Forecasting US output growth and inflation," Economic Research Papers 269743, University of Warwick - Department of Economics.
    19. Jin, Sainan & Corradi, Valentina & Swanson, Norman R., 2017. "Robust Forecast Comparison," Econometric Theory, Cambridge University Press, vol. 33(6), pages 1306-1351, December.
    20. Wagner Piazza Gaglianone & João Victor Issler & Silvia Maria Matos, 2017. "Applying a microfounded-forecasting approach to predict Brazilian inflation," Empirical Economics, Springer, vol. 53(1), pages 137-163, August.

    More about this item

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C51 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Construction and Estimation
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • E27 - Macroeconomics and Monetary Economics - - Consumption, Saving, Production, Employment, and Investment - - - Forecasting and Simulation: Models and Applications
    • E37 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Forecasting and Simulation: Models and Applications

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tky:fseres:2010cf729. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CIRJE administrative office (email available below). General contact details of provider: https://edirc.repec.org/data/ritokjp.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.