IDEAS home Printed from https://ideas.repec.org/p/tin/wpaper/20100088.html
   My bibliography  Save this paper

Does Disagreement amongst Forecasters have Predictive Value?

Author

Listed:
  • Rianne Legerstee

    (Erasmus University Rotterdam)

  • Philip Hans Franses

    (Erasmus University Rotterdam)

Abstract

This discussion paper has led to a publication in 'Journal of Forecasting' , 2015, 34(4), 290-302. Forecasts from various experts are often used in macroeconomic forecasting models. Usually the focus is on the mean or median of the survey data. In the present study we adopt a different perspective on the survey data as we examine the predictive power of disagreement amongst forecasters. The premise is that this variable could signal upcoming structural or temporal changes in an economic process or in the predictive power of the survey forecasts. In our empirical work, we examine a variety of macroeconomic variables, and we use different measurements for the degree of disagreement, together with measures for location of the survey data and autoregressive components. Forecasts from simple linear models and forecasts from Markov regime-switching models with constant and with time-varying transition probabilities are constructed in real-time and compared on forecast accuracy. We find that disagreement has predictive power indeed and that this variable can be used to improve forecasts when used in Markov regime-switching models.

Suggested Citation

  • Rianne Legerstee & Philip Hans Franses, 2010. "Does Disagreement amongst Forecasters have Predictive Value?," Tinbergen Institute Discussion Papers 10-088/4, Tinbergen Institute.
  • Handle: RePEc:tin:wpaper:20100088
    as

    Download full text from publisher

    File URL: https://papers.tinbergen.nl/10088.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
    2. Carlos Capistr¡N & Allan Timmermann, 2009. "Disagreement and Biases in Inflation Expectations," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 41(2-3), pages 365-396, March.
    3. Timmermann, Allan, 2000. "Moments of Markov switching models," Journal of Econometrics, Elsevier, vol. 96(1), pages 75-111, May.
    4. Victor Zarnowitz & Phillip Braun, 1993. "Twenty-two Years of the NBER-ASA Quarterly Economic Outlook Surveys: Aspects and Comparisons of Forecasting Performance," NBER Chapters, in: Business Cycles, Indicators, and Forecasting, pages 11-94, National Bureau of Economic Research, Inc.
    5. N. Gregory Mankiw & Ricardo Reis & Justin Wolfers, 2004. "Disagreement about Inflation Expectations," NBER Chapters, in: NBER Macroeconomics Annual 2003, Volume 18, pages 209-270, National Bureau of Economic Research, Inc.
    6. Jonas Dovern & Ulrich Fritsche & Jiri Slacalek, 2012. "Disagreement Among Forecasters in G7 Countries," The Review of Economics and Statistics, MIT Press, vol. 94(4), pages 1081-1096, November.
    7. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    8. Webby, Richard & O'Connor, Marcus, 1996. "Judgemental and statistical time series forecasting: a review of the literature," International Journal of Forecasting, Elsevier, vol. 12(1), pages 91-118, March.
    9. Graham Elliott & Allan Timmermann, 2005. "Optimal Forecast Combination Under Regime Switching ," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 46(4), pages 1081-1102, November.
    10. Lamont, Owen A., 2002. "Macroeconomic forecasts and microeconomic forecasters," Journal of Economic Behavior & Organization, Elsevier, vol. 48(3), pages 265-280, July.
    11. David Laster & Paul Bennett & In Sun Geoum, 1999. "Rational Bias in Macroeconomic Forecasts," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(1), pages 293-318.
    12. Hamilton, James D, 1989. "A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle," Econometrica, Econometric Society, vol. 57(2), pages 357-384, March.
    13. Victor Zarnowitz & Louis A. Lambros, 1983. "Consensus and Uncertainty in Economic Prediction," NBER Working Papers 1171, National Bureau of Economic Research, Inc.
    14. James H. Stock & Mark W. Watson, 1993. "Business Cycles, Indicators, and Forecasting," NBER Books, National Bureau of Economic Research, Inc, number stoc93-1, March.
    15. Smith, Aaron & Naik, Prasad A. & Tsai, Chih-Ling, 2006. "Markov-switching model selection using Kullback-Leibler divergence," Journal of Econometrics, Elsevier, vol. 134(2), pages 553-577, October.
    16. Clemen, Robert T., 1989. "Combining forecasts: A review and annotated bibliography," International Journal of Forecasting, Elsevier, vol. 5(4), pages 559-583.
    17. Aurelien Baillon & Laure Cabantous, 2009. "Combining imprecise or conflicting probability judgments: A choice-based study," ICBBR Working Papers 3, International Centre for Behavioural Business Research.
    18. Francis X. Diebold & Jose A. Lopez, 1995. "Forecast evaluation and combination," Research Paper 9525, Federal Reserve Bank of New York.
    19. Mc Cracken, Michael W., 2000. "Robust out-of-sample inference," Journal of Econometrics, Elsevier, vol. 99(2), pages 195-223, December.
    20. Stock, James H. & Watson, Mark W. (ed.), 1993. "Business Cycles, Indicators, and Forecasting," National Bureau of Economic Research Books, University of Chicago Press, edition 1, number 9780226774886, December.
    21. Kajal Lahiri & Xuguang Sheng, 2010. "Measuring forecast uncertainty by disagreement: The missing link," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 514-538.
    22. Harvey, David & Leybourne, Stephen & Newbold, Paul, 1997. "Testing the equality of prediction mean squared errors," International Journal of Forecasting, Elsevier, vol. 13(2), pages 281-291, June.
    23. Zacharias Psaradakis & Nicola Spagnolo, 2006. "Joint Determination of the State Dimension and Autoregressive Order for Models with Markov Regime Switching," Journal of Time Series Analysis, Wiley Blackwell, vol. 27(5), pages 753-766, September.
    24. Zarnowitz, Victor & Lambros, Louis A, 1987. "Consensus and Uncertainty in Economic Prediction," Journal of Political Economy, University of Chicago Press, vol. 95(3), pages 591-621, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Badarinza, Cristian & Gross, Marco, 2011. "Macroeconomic vulnerability and disagreement in expectations," Working Paper Series 1407, European Central Bank.
    2. repec:amu:wpaper:2013-04 is not listed on IDEAS
    3. Philip Hans Franses & Max Welz, 2022. "Evaluating heterogeneous forecasts for vintages of macroeconomic variables," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(4), pages 829-839, July.
    4. Philip Hans Franses, 2020. "Correcting the January optimism effect," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 39(6), pages 927-933, September.
    5. Franses, Ph.H.B.F. & Maassen, N.R., 2015. "Consensus forecasters: How good are they individually and why?," Econometric Institute Research Papers EI2015-21, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    6. Philip Hans Franses, 2021. "Modeling Judgment in Macroeconomic Forecasts," Journal of Quantitative Economics, Springer;The Indian Econometric Society (TIES), vol. 19(1), pages 401-417, December.
    7. Nautz, Dieter & Pagenhardt, Laura & Strohsal, Till, 2017. "The (de-)anchoring of inflation expectations: New evidence from the euro area," The North American Journal of Economics and Finance, Elsevier, vol. 40(C), pages 103-115.
    8. Franses, Ph.H.B.F., 2019. "Professional Forecasters and January," Econometric Institute Research Papers EI2019-25, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    9. Félix, Luiz & Kräussl, Roman & Stork, Philip, 2018. "Predictable biases in macroeconomic forecasts and their impact across asset classes," CFS Working Paper Series 596, Center for Financial Studies (CFS).
    10. Luiz Félix & Roman Kräussl & Philip Stork, 2021. "Strategic bias and popularity effect in the prediction of economic surprises," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 40(6), pages 1095-1117, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Constantin Burgi, 2016. "What Do We Lose When We Average Expectations?," Working Papers 2016-013, The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting.
    2. Lahiri, Kajal & Sheng, Xuguang, 2010. "Learning and heterogeneity in GDP and inflation forecasts," International Journal of Forecasting, Elsevier, vol. 26(2), pages 265-292, April.
    3. Harvey, David I. & Newbold, Paul, 2003. "The non-normality of some macroeconomic forecast errors," International Journal of Forecasting, Elsevier, vol. 19(4), pages 635-653.
    4. Clements, Michael P., 2010. "Explanations of the inconsistencies in survey respondents' forecasts," European Economic Review, Elsevier, vol. 54(4), pages 536-549, May.
    5. Scott Schuh, 2001. "An evaluation of recent macroeconomic forecast errors," New England Economic Review, Federal Reserve Bank of Boston, pages 35-56.
    6. Magnus Reif, 2020. "Macroeconomics, Nonlinearities, and the Business Cycle," ifo Beiträge zur Wirtschaftsforschung, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, number 87.
    7. McCracken,M.W. & West,K.D., 2001. "Inference about predictive ability," Working papers 14, Wisconsin Madison - Social Systems.
    8. Paul Bennett & In Sun Geoum & David S. Laster, 1997. "Rational bias in macroeconomic forecasts," Staff Reports 21, Federal Reserve Bank of New York.
    9. Gavin, William T. & Mandal, Rachel J., 2003. "Evaluating FOMC forecasts," International Journal of Forecasting, Elsevier, vol. 19(4), pages 655-667.
    10. Guidolin, Massimo & Timmermann, Allan, 2009. "Forecasts of US short-term interest rates: A flexible forecast combination approach," Journal of Econometrics, Elsevier, vol. 150(2), pages 297-311, June.
    11. Kajal Lahiri & Fushang Liu, 2006. "Modelling multi‐period inflation uncertainty using a panel of density forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 21(8), pages 1199-1219, December.
    12. Ottaviani, Marco & Sorensen, Peter Norman, 2006. "The strategy of professional forecasting," Journal of Financial Economics, Elsevier, vol. 81(2), pages 441-466, August.
    13. Francis X. Diebold & Glenn D. Rudebusch, 2001. "Five questions about business cycles," Economic Review, Federal Reserve Bank of San Francisco, pages 1-15.
    14. Hubrich, Kirstin, 2005. "Forecasting euro area inflation: Does aggregating forecasts by HICP component improve forecast accuracy?," International Journal of Forecasting, Elsevier, vol. 21(1), pages 119-136.
    15. Massimo Guidolin & Carrie Fangzhou Na, 2007. "The economic and statistical value of forecast combinations under regime switching: an application to predictable U.S. returns," Working Papers 2006-059, Federal Reserve Bank of St. Louis.
    16. Lamont, Owen A., 2002. "Macroeconomic forecasts and microeconomic forecasters," Journal of Economic Behavior & Organization, Elsevier, vol. 48(3), pages 265-280, July.
    17. Alexander, Marcus & Christakis, Nicholas A., 2008. "Bias and asymmetric loss in expert forecasts: A study of physician prognostic behavior with respect to patient survival," Journal of Health Economics, Elsevier, vol. 27(4), pages 1095-1108, July.
    18. Papastamos, Dimitrios & Matysiak, George & Stevenson, Simon, 2015. "Assessing the accuracy and dispersion of real estate investment forecasts," International Review of Financial Analysis, Elsevier, vol. 42(C), pages 141-152.
    19. Hartmann, Matthias & Herwartz, Helmut & Ulm, Maren, 2017. "A comparative assessment of alternative ex ante measures of inflation uncertainty," International Journal of Forecasting, Elsevier, vol. 33(1), pages 76-89.
    20. Kim, Hyun Hak & Swanson, Norman R., 2018. "Mining big data using parsimonious factor, machine learning, variable selection and shrinkage methods," International Journal of Forecasting, Elsevier, vol. 34(2), pages 339-354.

    More about this item

    Keywords

    model forecasts; expert forecasts; survey forecasts; Markov regime-switching models; disagreement; time series;
    All these keywords.

    JEL classification:

    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tin:wpaper:20100088. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Tinbergen Office +31 (0)10-4088900 (email available below). General contact details of provider: https://edirc.repec.org/data/tinbenl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.