IDEAS home Printed from https://ideas.repec.org/p/rdg/icmadp/icma-dp2016-08.html
   My bibliography  Save this paper

Are Macro-Forecasters Essentially The Same? An Analysis of Disagreement, Accuracy and Efficiency

Author

Listed:
  • Michael Clements

    (ICMA Centre, Henley Business School, University of Reading)

Abstract

We investigate whether there are systematic differences between forecasters in terms of their levels of disagreement and the accuracy of their forecasts, and whether these differences are related to whether or not a forecaster efficiently uses their available information. We find that forecasters are not interchangeable. At any point in time, the level of disagreement between forecasters is more likely to be due to a given set of forecasters, as opposed to any randomly-selected set of forecasters. In terms of forecast accuracy, we also find persistence, in that forecasters who are more (less) accurate in one period tend to be more (less) accurate in a subsequent period. Finally, we reject efficiency for around half of all forecasters at short horizons (depending on the variable in question), and ?find that efficient forecasters tend to be more accurate and less contrarian. Our results do not support the notion that contrarian forecasts stand apart by virtue of having superior information - knowing something that others do not.

Suggested Citation

  • Michael Clements, 2016. "Are Macro-Forecasters Essentially The Same? An Analysis of Disagreement, Accuracy and Efficiency," ICMA Centre Discussion Papers in Finance icma-dp2016-08, Henley Business School, University of Reading.
  • Handle: RePEc:rdg:icmadp:icma-dp2016-08
    as

    Download full text from publisher

    File URL: http://www.henley.ac.uk/files/pdf/exec-ed/ICM-2016-08.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ottaviani, Marco & Sorensen, Peter Norman, 2006. "The strategy of professional forecasting," Journal of Financial Economics, Elsevier, vol. 81(2), pages 441-466, August.
    2. Jacob A. Mincer & Victor Zarnowitz, 1969. "The Evaluation of Economic Forecasts," NBER Chapters, in: Economic Forecasts and Expectations: Analysis of Forecasting Behavior and Performance, pages 3-46, National Bureau of Economic Research, Inc.
    3. Olivier Coibion & Yuriy Gorodnichenko, 2012. "What Can Survey Forecasts Tell Us about Information Rigidities?," Journal of Political Economy, University of Chicago Press, vol. 120(1), pages 116-159.
    4. Andrade, Philippe & Le Bihan, Hervé, 2013. "Inattentive professional forecasters," Journal of Monetary Economics, Elsevier, vol. 60(8), pages 967-982.
    5. Ivana Komunjer & Michael T. Owyang, 2012. "Multivariate Forecast Evaluation and Rationality Testing," The Review of Economics and Statistics, MIT Press, vol. 94(4), pages 1066-1080, November.
    6. Hendry, David F. & Martinez, Andrew B., 2017. "Evaluating multi-step system forecasts with relatively few forecast-error observations," International Journal of Forecasting, Elsevier, vol. 33(2), pages 359-372.
    7. Croushore, Dean & Stark, Tom, 2001. "A real-time data set for macroeconomists," Journal of Econometrics, Elsevier, vol. 105(1), pages 111-130, November.
    8. N. Gregory Mankiw & Ricardo Reis, 2002. "Sticky Information versus Sticky Prices: A Proposal to Replace the New Keynesian Phillips Curve," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 117(4), pages 1295-1328.
    9. Andrew Patton & Allan Timmermann, 2012. "Forecast Rationality Tests Based on Multi-Horizon Bounds," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(1), pages 1-17.
    10. Antonello D’Agostino & Kieran Mcquinn & Karl Whelan, 2012. "Are Some Forecasters Really Better Than Others?," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 44(4), pages 715-732, June.
    11. Andrade, Philippe & Crump, Richard K. & Eusepi, Stefano & Moench, Emanuel, 2016. "Fundamental disagreement," Journal of Monetary Economics, Elsevier, vol. 83(C), pages 106-128.
    12. Michael P. Clements, 2014. "Forecast Uncertainty- Ex Ante and Ex Post : U.S. Inflation and Output Growth," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 32(2), pages 206-216, April.
    13. Whelan, Karl, 2003. "A Two-Sector Approach to Modeling U.S. NIPA Data," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 35(4), pages 627-656, August.
    14. Rich, Robert W & Butler, J S, 1998. "Disagreement as a Measure of Uncertainty: A Comment on Bomberger," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 30(3), pages 411-419, August.
    15. Stark, Tom & Croushore, Dean, 2002. "Forecasting with a real-time data set for macroeconomists," Journal of Macroeconomics, Elsevier, vol. 24(4), pages 507-531, December.
    16. Lamont, Owen A., 2002. "Macroeconomic forecasts and microeconomic forecasters," Journal of Economic Behavior & Organization, Elsevier, vol. 48(3), pages 265-280, July.
    17. Davies, Anthony & Lahiri, Kajal, 1995. "A new framework for analyzing survey forecasts using three-dimensional panel data," Journal of Econometrics, Elsevier, vol. 68(1), pages 205-227, July.
    18. Ball, Laurence & Jalles, João Tovar & Loungani, Prakash, 2015. "Do forecasters believe in Okun’s Law? An assessment of unemployment and output forecasts," International Journal of Forecasting, Elsevier, vol. 31(1), pages 176-184.
    19. L. R. Klein & R. F. Kosobud, 1961. "Some Econometrics of Growth: Great Ratios of Economics," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 75(2), pages 173-198.
    20. David Laster & Paul Bennett & In Sun Geoum, 1999. "Rational Bias in Macroeconomic Forecasts," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(1), pages 293-318.
    21. Olivier Coibion & Yuriy Gorodnichenko, 2015. "Information Rigidity and the Expectations Formation Process: A Simple Framework and New Facts," American Economic Review, American Economic Association, vol. 105(8), pages 2644-2678, August.
    22. Raffaella Giacomini & Vasiliki Skreta & Javier Turen, 2015. "Models, Inattention and Expectation Updates," Discussion Papers 1602, Centre for Macroeconomics (CFM).
    23. Clements, Michael P., 2016. "Long-run restrictions and survey forecasts of output, consumption and investment," International Journal of Forecasting, Elsevier, vol. 32(3), pages 614-628.
    24. Bomberger, William A, 1996. "Disagreement as a Measure of Uncertainty," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 28(3), pages 381-392, August.
    25. repec:awi:wpaper:0571 is not listed on IDEAS
    26. Dean Croushore, 2011. "Frontiers of Real-Time Data Analysis," Journal of Economic Literature, American Economic Association, vol. 49(1), pages 72-100, March.
    27. J. Steven Landefeld & Eugene P. Seskin & Barbara M. Fraumeni, 2008. "Taking the Pulse of the Economy: Measuring GDP," Journal of Economic Perspectives, American Economic Association, vol. 22(2), pages 193-216, Spring.
    28. Lahiri, Kajal & Sheng, Xuguang, 2008. "Evolution of forecast disagreement in a Bayesian learning model," Journal of Econometrics, Elsevier, vol. 144(2), pages 325-340, June.
    29. Robert Rich & Joseph Tracy, 2010. "The Relationships among Expected Inflation, Disagreement, and Uncertainty: Evidence from Matched Point and Density Forecasts," The Review of Economics and Statistics, MIT Press, vol. 92(1), pages 200-207, February.
    30. Sims, Christopher A., 2003. "Implications of rational inattention," Journal of Monetary Economics, Elsevier, vol. 50(3), pages 665-690, April.
    31. Patton, Andrew J. & Timmermann, Allan, 2010. "Why do forecasters disagree? Lessons from the term structure of cross-sectional dispersion," Journal of Monetary Economics, Elsevier, vol. 57(7), pages 803-820, October.
    32. Zarnowitz, Victor & Lambros, Louis A, 1987. "Consensus and Uncertainty in Economic Prediction," Journal of Political Economy, University of Chicago Press, vol. 95(3), pages 591-621, June.
    33. Clements, Michael P, 1995. "Rationality and the Role of Judgement in Macroeconomic Forecasting," Economic Journal, Royal Economic Society, vol. 105(429), pages 410-420, March.
    34. Chanont Banternghansa & Michael W. McCracken, 2009. "Forecast disagreement among FOMC members," Working Papers 2009-059, Federal Reserve Bank of St. Louis.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michael P. Clements, 2022. "Forecaster Efficiency, Accuracy, and Disagreement: Evidence Using Individual‐Level Survey Data," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 54(2-3), pages 537-568, March.
    2. Clements, Michael P., 2019. "Do forecasters target first or later releases of national accounts data?," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1240-1249.
    3. Clements, Michael P., 2021. "Do survey joiners and leavers differ from regular participants? The US SPF GDP growth and inflation forecasts," International Journal of Forecasting, Elsevier, vol. 37(2), pages 634-646.
    4. Michael P Clements, 2014. "Assessing the Evidence of Macro- Forecaster Herding: Forecasts of Inflation and Output Growth," ICMA Centre Discussion Papers in Finance icma-dp2014-12, Henley Business School, University of Reading.
    5. Michael P. Clements, 2014. "US Inflation Expectations and Heterogeneous Loss Functions, 1968–2010," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 33(1), pages 1-14, January.
    6. repec:zbw:bofrdp:037 is not listed on IDEAS
    7. Ambrocio, Gene, 2017. "The real effects of overconfidence and fundamental uncertainty shocks," Research Discussion Papers 37/2017, Bank of Finland.
    8. repec:zbw:bofrdp:2017_037 is not listed on IDEAS
    9. Clements, Michael P., 2010. "Explanations of the inconsistencies in survey respondents' forecasts," European Economic Review, Elsevier, vol. 54(4), pages 536-549, May.
    10. An, Zidong & Zheng, Xinye, 2023. "Diligent forecasters can make accurate predictions despite disagreeing with the consensus," Economic Modelling, Elsevier, vol. 125(C).
    11. Manzan, Sebastiano, 2021. "Are professional forecasters Bayesian?," Journal of Economic Dynamics and Control, Elsevier, vol. 123(C).
    12. Dovern, Jonas & Jannsen, Nils, 2017. "Systematic errors in growth expectations over the business cycle," International Journal of Forecasting, Elsevier, vol. 33(4), pages 760-769.
    13. Clements, Michael P., 2006. "Internal consistency of survey respondentsíforecasts: Evidence based on the Survey of Professional Forecasters," Economic Research Papers 269742, University of Warwick - Department of Economics.
    14. Clements, Michael P., 2012. "Do professional forecasters pay attention to data releases?," International Journal of Forecasting, Elsevier, vol. 28(2), pages 297-308.
    15. Constantin Burgi, 2016. "What Do We Lose When We Average Expectations?," Working Papers 2016-013, The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting.
    16. Andrade, Philippe & Crump, Richard K. & Eusepi, Stefano & Moench, Emanuel, 2016. "Fundamental disagreement," Journal of Monetary Economics, Elsevier, vol. 83(C), pages 106-128.
    17. Chini, Emilio Zanetti, 2023. "Can we estimate macroforecasters’ mis-behavior?," Journal of Economic Dynamics and Control, Elsevier, vol. 149(C).
    18. Michael P. Clements, 2018. "Do Macroforecasters Herd?," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 50(2-3), pages 265-292, March.
    19. Richard K. Crump & Stefano Eusepi & Emanuel Moench & Bruce Preston, 2021. "The Term Structure of Expectations," Staff Reports 992, Federal Reserve Bank of New York.
    20. Cole, Stephen J. & Milani, Fabio, 2021. "Heterogeneity in individual expectations, sentiment, and constant-gain learning," Journal of Economic Behavior & Organization, Elsevier, vol. 188(C), pages 627-650.
    21. Broer, Tobias & Kohlhas, Alexandre, 2018. "Forecaster (Mis-)Behavior," CEPR Discussion Papers 12898, C.E.P.R. Discussion Papers.
    22. Czudaj, Robert L., 2022. "Heterogeneity of beliefs and information rigidity in the crude oil market: Evidence from survey data," European Economic Review, Elsevier, vol. 143(C).

    More about this item

    Keywords

    Expectations formation; Disagreement; Accuracy; Forecast Efficiency;
    All these keywords.

    JEL classification:

    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • E37 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Forecasting and Simulation: Models and Applications

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:rdg:icmadp:icma-dp2016-08. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Marie Pearson (email available below). General contact details of provider: https://edirc.repec.org/data/bsrdguk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.