Quantifying survey expectations: What's wrong with the probability approach?
We study a matched sample of individual stock market forecasts consisting of both qualitative and quantitative forecasts. This allows us to test for the quality of forecast quantification methods by comparing quantified qualitative forecasts with actual quantitative forecasts. Focusing mainly on the widely used quantification framework advocated by Carlson and Parkin (1975), the so-called "probability approach", we find that quantified expectations derived from the probability approach display a surprisingly weak correlation with reported quantitative stock return forecasts. We trace the reason for this low correlation to the importance of asymmetric and time-varying thresholds, whereas individual heterogeneity across forecasters seems to play a minor role. Hence, our results suggest that qualitative survey data may not be a very useful device to obtain quantitative forecasts and we suggest ways to remedy this problem when designing qualitative surveys.
|Date of creation:||Dec 2011|
|Date of revision:|
|Contact details of provider:|| Postal: Koenigsworther Platz 1, D-30167 Hannover|
Phone: (0511) 762-5350
Fax: (0511) 762-5665
Web page: http://www.wiwi.uni-hannover.de
More information through EDIRC
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Newey, Whitney & West, Kenneth, 2014.
"A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix,"
Publishing House "SINERGIA PRESS", vol. 33(1), pages 125-132.
- Newey, Whitney K & West, Kenneth D, 1987. "A Simple, Positive Semi-definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix," Econometrica, Econometric Society, vol. 55(3), pages 703-08, May.
- Whitney K. Newey & Kenneth D. West, 1986. "A Simple, Positive Semi-Definite, Heteroskedasticity and AutocorrelationConsistent Covariance Matrix," NBER Technical Working Papers 0055, National Bureau of Economic Research, Inc.
- Mankiw, N. Gregory & Reis, Ricardo & Wolfers, Justin, 2003.
"Disagreement about Inflation Expectations,"
1807, Stanford University, Graduate School of Business.
- N. Gregory Mankiw & Ricardo Reis & Justin Wolfers, 2003. "Disagreement about Inflation Expectations," NBER Working Papers 9796, National Bureau of Economic Research, Inc.
- N. Gregory Mankiw & Ricardo Augusto Marc Rocha Reis & Justin Wolfers, 2004. "Disagreement about Inflation Expectations," Yale School of Management Working Papers ysm391, Yale School of Management.
- N. Gregory Mankiw & Ricardo Reis & Justin Wolfers, 2003. "Disagreement about Inflation Expectations," Harvard Institute of Economic Research Working Papers 2011, Harvard - Institute of Economic Research.
- Mitchell, James & Mouratidis, Kostas & Weale, Martin, 2007. "Uncertainty in UK manufacturing: Evidence from qualitative survey data," Economics Letters, Elsevier, vol. 94(2), pages 245-252, February.
- Steffen Henzel & Timo Wollmershäuser, 2005. "An Alternative to the Carlson-Parkin Method for the Quantification of Qualitative Inflation Expectations: Evidence from the Ifo World Economic Survey," Ifo Working Paper Series Ifo Working Papers No. 9, Ifo Institute - Leibniz Institute for Economic Research at the University of Munich.
- Frankel, Jeffrey A & Froot, Kenneth A, 1990. "Chartists, Fundamentalists, and Trading in the Foreign Exchange Market," American Economic Review, American Economic Association, vol. 80(2), pages 181-85, May.
- Anderson, Evan W. & Ghysels, Eric & Juergens, Jennifer L., 2009. "The impact of risk and uncertainty on expected returns," Journal of Financial Economics, Elsevier, vol. 94(2), pages 233-263, November.
- Lui, Silvia & Mitchell, James & Weale, Martin, 2011.
"The utility of expectational data: Firm-level evidence using matched qualitative-quantitative UK surveys,"
International Journal of Forecasting,
Elsevier, vol. 27(4), pages 1128-1146, October.
- Dr Silvia Lui & Dr Martin Weale & Dr. James Mitchell, 2009. "The utility of expectational data: Firm-level evidence using matched qualitative-quantitative UK surveys," NIESR Discussion Papers 343, National Institute of Economic and Social Research.
- Dr Silvia Lui & Dr Martin Weale & Dr. James Mitchell, 2008.
"Qualitative Business Surveys: Signal or Noise?,"
NIESR Discussion Papers
323, National Institute of Economic and Social Research.
- Pesaran, M.H. & Weale, M., 2005.
Cambridge Working Papers in Economics
0536, Faculty of Economics, University of Cambridge.
- Thomas Maag, 2009. "On the Accuracy of the Probability Method for Quantifying Beliefs about Inflation," KOF Working papers 09-230, KOF Swiss Economic Institute, ETH Zurich.
- Batchelor, Roy A & Orr, Adrian B, 1988. "Inflation Expectations Revisited," Economica, London School of Economics and Political Science, vol. 55(219), pages 317-31, August.
- James Mitchell & Richard J. Smith & Martin R. Weale, 2002. "Quantification of Qualitative Firm-Level Survey Data," Economic Journal, Royal Economic Society, vol. 112(478), pages C117-C135, March.
- Graham Elliott & Allan Timmermann & Ivana Komunjer, 2005. "Estimation and Testing of Forecast Rationality under Flexible Loss," Review of Economic Studies, Oxford University Press, vol. 72(4), pages 1107-1125.
- Jörg Döpke & Jonas Dovern & Ulrich Fritsche & Jiri Slacalek, 2006.
"The Dynamics of European Inflation Expectations,"
Discussion Papers of DIW Berlin
571, DIW Berlin, German Institute for Economic Research.
- Patton, Andrew J. & Timmermann, Allan, 2010. "Why do forecasters disagree? Lessons from the term structure of cross-sectional dispersion," Journal of Monetary Economics, Elsevier, vol. 57(7), pages 803-820, October.
- Jörg Breitung, 2008. "Assessing the Rationality of Survey Expectations: The Probability Approach," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), Justus-Liebig University Giessen, Department of Statistics and Economics, vol. 228(5+6), pages 630-643, December.
- Fishe, Raymond P. H. & Lahiri, Kajal, 1981. "On the estimation of inflationary expectations from qualitative responses," Journal of Econometrics, Elsevier, vol. 16(1), pages 89-102, May.
- Menkhoff, Lukas & Rebitzky, Rafael R. & Schröder, Michael, 2009.
"Heterogeneity in exchange rate expectations: Evidence on the chartist-fundamentalist approach,"
Journal of Economic Behavior & Organization,
Elsevier, vol. 70(1-2), pages 241-252, May.
- Lukas Menkhoff & Rafael R. Rebitzky & Michael Schröder, 2008. "Heterogeneity in Exchange Rate Expectations: Evidence on the Chartist-Fundamentalist Approach," CESifo Working Paper Series 2502, CESifo Group Munich.
- Carlson, John A & Parkin, J Michael, 1975. "Inflation Expectations," Economica, London School of Economics and Political Science, vol. 42(166), pages 123-38, May.
- Batchelor, R A, 1986. "Quantitative v. Qualitative Measures of Inflation Expectations," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 48(2), pages 99-120, May.
- Defris, L. V. & Williams, R. A., 1979. "Quantitative versus qualitative measures of price expectations: The evidence from Australian consumer surveys," Economics Letters, Elsevier, vol. 2(2), pages 169-173.
- James Mitchell & Richard J. Smith & Martin R. Weale, 2005. "Forecasting Manufacturing Output Growth Using Firm-Level Survey Data," Manchester School, University of Manchester, vol. 73(4), pages 479-499, 07.
- Claveria, Oscar & Pons, Ernest & Ramos, Raul, 2007. "Business and consumer expectations and macroeconomic forecasts," International Journal of Forecasting, Elsevier, vol. 23(1), pages 47-69.
- Mitchell, James, 2002. "The use of non-normal distributions in quantifying qualitative survey data on expectations," Economics Letters, Elsevier, vol. 76(1), pages 101-107, June.
- Michela Nardo, 2003. "The Quantification of Qualitative Survey Data : A Critical Assessment," Journal of Economic Surveys, Wiley Blackwell, vol. 17(5), pages 645-668, December.
- Pesaran, M Hashem, 1985. "Formation of Inflation Expectations in British Manufacturing Industries," Economic Journal, Royal Economic Society, vol. 95(380), pages 948-75, December.
- Batchelor, R. A., 1982. "Expectations, output and inflation : The European experience," European Economic Review, Elsevier, vol. 17(1), pages 1-25.
- Müller, Christian, 2010. "You CAN Carlson-Parkin," Economics Letters, Elsevier, vol. 108(1), pages 33-35, July.
- Dasgupta, Susmita & Lahiri, Kajal, 1992. "A Comparative Study of Alternative Methods of Quantifying Qualitative Survey Responses Using NAPM Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 10(4), pages 391-400, October.
- Nolte, Ingmar & Pohlmeier, Winfried, 2007. "Using forecasts of forecasters to forecast," International Journal of Forecasting, Elsevier, vol. 23(1), pages 15-28.
- Jan Marc Berk, 1999. "Measuring inflation expectations: a survey data approach," Applied Economics, Taylor & Francis Journals, vol. 31(11), pages 1467-1480.
When requesting a correction, please mention this item's handle: RePEc:han:dpaper:dp-485. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Heidrich, Christian)
If references are entirely missing, you can add them using this form.