IDEAS home Printed from https://ideas.repec.org/a/bla/jecsur/v28y2014i2p195-208.html
   My bibliography  Save this article

Evaluating Macroeconomic Forecasts: A Concise Review Of Some Recent Developments

Author

Listed:
  • Philip Hans Franses
  • Michael McAleer
  • Rianne Legerstee

Abstract

Macroeconomic forecasts are frequently produced, widely published, intensively discussed, and comprehensively used. The formal evaluation of such forecasts has a long research history. Recently, a new angle to the evaluation of forecasts has been addressed, and in this review we analyze some recent developments from that perspective. The literature on forecast evaluation predominantly assumes that macroeconomic forecasts are generated from econometric models. In practice, however, most macroeconomic forecasts, such as those from the IMF, World Bank, OECD, Federal Reserve Board, Federal Open Market Committee (FOMC), and the ECB, are typically based on econometric model forecasts jointly with human intuition. This seemingly inevitable combination renders most of these forecasts biased and, as such, their evaluation becomes nonstandard. In this review, we consider the evaluation of two forecasts in which: (i) the two forecasts are generated from two distinct econometric models; (ii) one forecast is generated from an econometric model and the other is obtained as a combination of a model and intuition; and (iii) the two forecasts are generated from two distinct (but unknown) combinations of different models and intuition. It is shown that alternative tools are needed to compare and evaluate the forecasts in each of these three situations. These alternative techniques are illustrated by comparing the forecasts from the (econometric) Staff of the Federal Reserve Board and the FOMC on inflation, unemployment, and real GDP growth. It is shown that the FOMC does not forecast significantly better than the Staff, and that the intuition of the FOMC does not add significantly in forecasting the actual values of the economic fundamentals. This would seem to belie the purported expertise of the FOMC.

Suggested Citation

  • Philip Hans Franses & Michael McAleer & Rianne Legerstee, 2014. "Evaluating Macroeconomic Forecasts: A Concise Review Of Some Recent Developments," Journal of Economic Surveys, Wiley Blackwell, vol. 28(2), pages 195-208, April.
  • Handle: RePEc:bla:jecsur:v:28:y:2014:i:2:p:195-208
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1111/joes.12000
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Christina D. Romer & David H. Romer, 2004. "A New Measure of Monetary Shocks: Derivation and Implications," American Economic Review, American Economic Association, vol. 94(4), pages 1055-1084, September.
    2. Eroglu, Cuneyt & Croxton, Keely L., 2010. "Biases in judgmental adjustments of statistical forecasts: The role of individual differences," International Journal of Forecasting, Elsevier, vol. 26(1), pages 116-133, January.
    3. Christina D. Romer & David H. Romer, 2008. "The FOMC versus the Staff: Where Can Monetary Policymakers Add Value?," American Economic Review, American Economic Association, vol. 98(2), pages 230-235, May.
    4. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
    5. Philip Hans Franses & Michael McAleer & Rianne Legerstee, 2009. "Expert opinion versus expertise in forecasting," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 63(3), pages 334-346, August.
    6. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    7. Heij, Christiaan & van Dijk, Dick & Groenen, Patrick J.F., 2011. "Real-time macroeconomic forecasting with leading indicators: An empirical comparison," International Journal of Forecasting, Elsevier, vol. 27(2), pages 466-481, April.
    8. Batchelor, Roy, 2007. "Bias in macroeconomic forecasts," International Journal of Forecasting, Elsevier, vol. 23(2), pages 189-203.
    9. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    10. Graham Elliott & Allan Timmermann, 2016. "Economic Forecasting," Economics Books, Princeton University Press, edition 1, number 10740.
    11. Yock Y. Chong & David F. Hendry, 1986. "Econometric Evaluation of Linear Macro-Economic Models," Review of Economic Studies, Oxford University Press, vol. 53(4), pages 671-690.
    12. Granger, C. W. J. & Newbold, Paul, 1986. "Forecasting Economic Time Series," Elsevier Monographs, Elsevier, edition 2, number 9780122951831 edited by Shell, Karl.
    13. McAleer, Michael, 1992. "Efficient Estimation: The Rao-Zyskind Condition, Kruskal's Theorem and Ordinary Least Squares," The Economic Record, The Economic Society of Australia, vol. 68(200), pages 65-72, March.
    14. Chang, Chia-Lin & Franses, Philip Hans & McAleer, Michael, 2011. "How accurate are government forecasts of economic fundamentals? The case of Taiwan," International Journal of Forecasting, Elsevier, vol. 27(4), pages 1066-1075, October.
    15. Franses, Philip Hans & Kranendonk, Henk C. & Lanser, Debby, 2011. "One model and various experts: Evaluating Dutch macroeconomic forecasts," International Journal of Forecasting, Elsevier, vol. 27(2), pages 482-495.
    16. James H. Stock & Mark W.Watson, 2003. "Forecasting Output and Inflation: The Role of Asset Prices," Journal of Economic Literature, American Economic Association, vol. 41(3), pages 788-829, September.
    17. Roy Batchelor, 2007. "Forecaster Behaviour and Bias in Macroeconomic Forecasts," ifo Working Paper Series 39, ifo Institute - Leibniz Institute for Economic Research at the University of Munich.
    18. G. Elliott & C. Granger & A. Timmermann (ed.), 2006. "Handbook of Economic Forecasting," Handbook of Economic Forecasting, Elsevier, edition 1, volume 1, number 1.
    19. Franses, Philip Hans & Legerstee, Rianne, 2009. "Properties of expert adjustments on model-based SKU-level forecasts," International Journal of Forecasting, Elsevier, vol. 25(1), pages 35-47.
    20. Fildes, Robert & Goodwin, Paul & Lawrence, Michael & Nikolopoulos, Konstantinos, 2009. "Effective forecasting and judgmental adjustments: an empirical evaluation and strategies for improvement in supply-chain planning," International Journal of Forecasting, Elsevier, vol. 25(1), pages 3-23.
    21. Fiebig, Denzil G. & McAleer, Michael & Bartels, Robert, 1992. "Properties of ordinary least squares estimators in regression models with nonspherical disturbances," Journal of Econometrics, Elsevier, vol. 54(1-3), pages 321-334.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. What I Learned Last Week
      by Dave Giles in Econometrics Beat: Dave Giles' Blog on 2012-10-13 09:19:00

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bratu Mihaela, 2013. "An Evaluation Of Usa Unemployment Rate Forecasts In Terms Of Accuracy And Bias. Empirical Methods To Improve The Forecasts Accuracy," Annals - Economy Series, Constantin Brancusi University, Faculty of Economics, vol. 1, pages 170-180, February.
    2. Miquel Clar-Lopez & Jordi López-Tamayo & Raúl Ramos, 2014. "Unemployment forecasts, time varying coefficient models and the Okun’s law in Spanish regions," Economics and Business Letters, Oviedo University Press, vol. 3(4), pages 247-262.
    3. Gibson, Heather D. & Hall, Stephen G. & Tavlas, George S., 2020. "Nonlinear forecast combinations: An example using euro-area real GDP growth," Journal of Economic Behavior & Organization, Elsevier, vol. 180(C), pages 579-589.
    4. Mihaela Simionescu (Bratu), 2014. "The Performance of Predictions Based on the Dobrescu Macromodel for the Romanian Economy," Journal for Economic Forecasting, Institute for Economic Forecasting, vol. 0(3), pages 179-195, October.
    5. Claeys, Peter & Cimadomo, Jacopo & Poplawski Ribeiro, Marcos, 2014. "How do financial institutions forecast sovereign spreads?," Working Paper Series 1750, European Central Bank.
    6. Simionescu Mihaela, 2015. "Kalman Filter or VAR Models to Predict Unemployment Rate in Romania?," Naše gospodarstvo/Our economy, Sciendo, vol. 61(3), pages 3-21, June.
    7. Mihaela Bratu, 2012. "A Strategy to Improve the Survey of Professional Forecasters (SPF) Predictions Using Bias-Corrected-Accelerated (BCA) Bootstrap Forecast Intervals," International Journal of Synergy and Research, ToKnowPress, vol. 1(2), pages 45-59.
    8. Mihaela Simionescu, 2015. "The Improvement of Unemployment Rate Predictions Accuracy," Prague Economic Papers, Prague University of Economics and Business, vol. 2015(3), pages 274-286.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Franses, Ph.H.B.F. & McAleer, M.J. & Legerstee, R., 2010. "Evaluating Macroeconomic Forecast: A Review of Some Recent Developments," Econometric Institute Research Papers EI 2010-19, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    2. Chang, Chia-Lin & Franses, Philip Hans & McAleer, Michael, 2011. "How accurate are government forecasts of economic fundamentals? The case of Taiwan," International Journal of Forecasting, Elsevier, vol. 27(4), pages 1066-1075, October.
    3. Constantin Burgi, 2016. "What Do We Lose When We Average Expectations?," Working Papers 2016-013, The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting.
    4. Rossi, Barbara, 2006. "Are Exchange Rates Really Random Walks? Some Evidence Robust To Parameter Instability," Macroeconomic Dynamics, Cambridge University Press, vol. 10(1), pages 20-38, February.
    5. Chang, Chia Lin & Franses, Philip Hans & Mcaleer, Michael, 2012. "Evaluating Individual and Mean Non-Replicable Forecasts," Journal for Economic Forecasting, Institute for Economic Forecasting, vol. 0(3), pages 22-43, September.
    6. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    7. Clark, Todd E. & West, Kenneth D., 2007. "Approximately normal tests for equal predictive accuracy in nested models," Journal of Econometrics, Elsevier, vol. 138(1), pages 291-311, May.
    8. Barbara Rossi & Atsushi Inoue, 2012. "Out-of-Sample Forecast Tests Robust to the Choice of Window Size," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(3), pages 432-453, April.
    9. Clark, Todd & McCracken, Michael, 2013. "Advances in Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1107-1201, Elsevier.
    10. Matheson, Troy D., 2008. "Phillips curve forecasting in a small open economy," Economics Letters, Elsevier, vol. 98(2), pages 161-166, February.
    11. Chang, Chia-Lin & Franses, Philip Hans & McAleer, Michael, 2013. "Are forecast updates progressive?," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 93(C), pages 9-18.
    12. Wagner Piazza Gaglianone & João Victor Issler & Silvia Maria Matos, 2017. "Applying a microfounded-forecasting approach to predict Brazilian inflation," Empirical Economics, Springer, vol. 53(1), pages 137-163, August.
    13. Raffaella Giacomini & Barbara Rossi, 2013. "Forecasting in macroeconomics," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 17, pages 381-408, Edward Elgar Publishing.
    14. Busetti, Fabio & Marcucci, Juri, 2013. "Comparing forecast accuracy: A Monte Carlo investigation," International Journal of Forecasting, Elsevier, vol. 29(1), pages 13-27.
    15. Daniel Borup & Jonas N. Eriksen & Mads M. Kjær & Martin Thyrsgaard, 2020. "Predicting bond return predictability," CREATES Research Papers 2020-09, Department of Economics and Business Economics, Aarhus University.
    16. Todd E. Clark & Michael W. McCracken, 2002. "Forecast-based model selection in the presence of structural breaks," Research Working Paper RWP 02-05, Federal Reserve Bank of Kansas City.
    17. Franses, Philip Hans, 2013. "Improving judgmental adjustment of model-based forecasts," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 93(C), pages 1-8.
    18. Qu, Ritong & Timmermann, Allan & Zhu, Yinchu, 2019. "Do Any Economists Have Superior Forecasting Skills?," CEPR Discussion Papers 14112, C.E.P.R. Discussion Papers.
    19. Chang, C-L. & Franses, Ph.H.B.F. & McAleer, M.J., 2009. "How Accurate are Government Forecast of Economic Fundamentals?," Econometric Institute Research Papers EI 2009-09, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    20. Norman R. Swanson & Weiqi Xiong, 2018. "Big data analytics in economics: What have we learned so far, and where should we go from here?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 51(3), pages 695-746, August.

    More about this item

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C51 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Construction and Estimation
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • E27 - Macroeconomics and Monetary Economics - - Consumption, Saving, Production, Employment, and Investment - - - Forecasting and Simulation: Models and Applications
    • E37 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Forecasting and Simulation: Models and Applications

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jecsur:v:28:y:2014:i:2:p:195-208. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0950-0804 .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0950-0804 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.