IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v27y2011i4p1128-1146.html
   My bibliography  Save this article

The utility of expectational data: Firm-level evidence using matched qualitative-quantitative UK surveys

Author

Listed:
  • Lui, Silvia
  • Mitchell, James
  • Weale, Martin

Abstract

Qualitative expectational data from business surveys are widely used to construct forecasts. However, based typically on evaluation at the macroeconomic level, doubts persist about the utility of these data. This paper evaluates the ability of the underlying firm-level expectations to anticipate subsequent outcomes. Importantly, this evaluation is not hampered by only having access to qualitative outcome data obtained from subsequent business surveys. Quantitative outcome data are also exploited. This required access to a unique panel dataset which matches firms' responses from the qualitative business survey with the same firms' quantitative replies to a different survey carried out by the national statistical office. Nonparametric tests then reveal an apparent paradox. Despite evidence that the qualitative and quantitative outcome data are related, we find that the expectational data offer rational forecasts of the qualitative but not the quantitative outcomes. We discuss the role of "discretisation" errors and the loss function in explaining this paradox.

Suggested Citation

  • Lui, Silvia & Mitchell, James & Weale, Martin, 2011. "The utility of expectational data: Firm-level evidence using matched qualitative-quantitative UK surveys," International Journal of Forecasting, Elsevier, vol. 27(4), pages 1128-1146, October.
  • Handle: RePEc:eee:intfor:v:27:y:2011:i:4:p:1128-1146
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169207011000094
    Download Restriction: Full text for ScienceDirect subscribers only

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Hansson, Jesper & Jansson, Per & Lof, Marten, 2005. "Business survey data: Do they help in forecasting GDP growth?," International Journal of Forecasting, Elsevier, vol. 21(2), pages 377-389.
    2. Ivaldi, Marc, 1992. "Survey Evidence on the Rationality of Expectations," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 7(3), pages 225-241, July-Sept.
    3. Brown, Bryan W & Maital, Shlomo, 1981. "What Do Economists Know? An Empirical Study of Experts' Expectations," Econometrica, Econometric Society, vol. 49(2), pages 491-504, March.
    4. Bonham, Carl S & Cohen, Richard H, 2001. "To Aggregate, Pool, or Neither: Testing the Rational-Expectations Hypothesis Using Survey Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(3), pages 278-291, July.
    5. Clements, Michael P., 2010. "Explanations of the inconsistencies in survey respondents' forecasts," European Economic Review, Elsevier, vol. 54(4), pages 536-549, May.
    6. Engelberg, Joseph & Manski, Charles F. & Williams, Jared, 2009. "Comparing the Point Predictions and Subjective Probability Distributions of Professional Forecasters," Journal of Business & Economic Statistics, American Statistical Association, vol. 27, pages 30-41.
    7. Pesaran, M. Hashem & Weale, Martin, 2006. "Survey Expectations," Handbook of Economic Forecasting, Elsevier.
    8. Carroll, Christopher D & Fuhrer, Jeffrey C & Wilcox, David W, 1994. "Does Consumer Sentiment Forecast Household Spending? If So, Why?," American Economic Review, American Economic Association, vol. 84(5), pages 1397-1408, December.
    9. Troy D. Matheson & James Mitchell & Brian Silverstone, 2010. "Nowcasting and predicting data revisions using panel survey data," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 29(3), pages 313-330.
    10. Easaw, Joshy Z. & Heravi, Saeed M., 2004. "Evaluating consumer sentiments as predictors of UK household consumption behavior: Are they accurate and useful?," International Journal of Forecasting, Elsevier, vol. 20(4), pages 671-681.
    11. Gourieroux, Christian & Monfort, Alain & Renault, Eric & Trognon, Alain, 1987. "Generalised residuals," Journal of Econometrics, Elsevier, vol. 34(1-2), pages 5-32.
    12. Patton, Andrew J. & Timmermann, Allan, 2007. "Properties of optimal forecasts under asymmetric loss and nonlinearity," Journal of Econometrics, Elsevier, vol. 140(2), pages 884-918, October.
    13. Ciaran Driver & Giovanni Urga, 2004. "Transforming Qualitative Survey Data: Performance Comparisons for the UK," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 66(1), pages 71-89, February.
    14. Wallis, Kenneth F, 1980. "Econometric Implications of the Rational Expectations Hypothesis," Econometrica, Econometric Society, vol. 48(1), pages 49-73, January.
    15. Zellner, Arnold, 1986. "Biased predictors, rationality and the evaluation of forecasts," Economics Letters, Elsevier, vol. 21(1), pages 45-48.
    16. Forni, Mario, et al, 2001. "Coincident and Leading Indicators for the Euro Area," Economic Journal, Royal Economic Society, vol. 111(471), pages 62-85, May.
    17. William A. Branch, 2004. "The Theory of Rationally Heterogeneous Expectations: Evidence from Survey Data on Inflation Expectations," Economic Journal, Royal Economic Society, vol. 114(497), pages 592-621, July.
    18. Souleles, Nicholas S, 2004. "Expectations, Heterogeneous Forecast Errors, and Consumption: Micro Evidence from the Michigan Consumer Sentiment Surveys," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 36(1), pages 39-72, February.
    19. Abberger, Klaus, 2007. "Qualitative business surveys and the assessment of employment -- A case study for Germany," International Journal of Forecasting, Elsevier, vol. 23(2), pages 249-258.
    20. James Mitchell & Richard J. Smith & Martin R. Weale, 2005. "Forecasting Manufacturing Output Growth Using Firm-Level Survey Data," Manchester School, University of Manchester, vol. 73(4), pages 479-499, July.
    21. Stock, James H & Watson, Mark W, 2002. "Macroeconomic Forecasting Using Diffusion Indexes," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 147-162, April.
    22. Gourieroux, Christian & Pradel, Jacqueline, 1986. "Direct test of the rational expectation hypothesis," European Economic Review, Elsevier, vol. 30(2), pages 265-284, April.
    23. Clements,Michael & Hendry,David, 1998. "Forecasting Economic Time Series," Cambridge Books, Cambridge University Press, number 9780521634809, March.
    24. Nerlove, Marc, 1983. "Expectations, Plans, and Realizations in Theory and Practice," Econometrica, Econometric Society, vol. 51(5), pages 1251-1279, September.
    25. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    26. McIntosh, James & Schiantarelli, Fabio & Low, William, 1989. "A Qualitative Response Analysis of UK Firms' Employment and Output Decisions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 4(3), pages 251-264, July-Sept.
    27. Keane, Michael P & Runkle, David E, 1990. "Testing the Rationality of Price Forecasts: New Evidence from Panel Data," American Economic Review, American Economic Association, vol. 80(4), pages 714-735, September.
    28. Sendhil Mullainathan & Marianne Bertrand, 2001. "Do People Mean What They Say? Implications for Subjective Survey Data," American Economic Review, American Economic Association, vol. 91(2), pages 67-72, May.
    29. Kauppi, Eija & Lassila, Jukka & Teräsvirta, Timo, 1996. "Short-Term Forecasting of Industrial Production with Business Survey Data: Experience from Finland's Great Depression," Discussion Papers 546, The Research Institute of the Finnish Economy.
    30. Carl S Bonham & Richard H Cohen, 2000. "Testing the Rational Expectations Hypothesis using Survey Data," Working Papers 200007, University of Hawaii at Manoa, Department of Economics.
    31. Patton, Andrew J. & Timmermann, Allan, 2007. "Testing Forecast Optimality Under Unknown Loss," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 1172-1184, December.
    32. Claveria, Oscar & Pons, Ernest & Ramos, Raul, 2007. "Business and consumer expectations and macroeconomic forecasts," International Journal of Forecasting, Elsevier, vol. 23(1), pages 47-69.
    33. Carlson, John A & Parkin, J Michael, 1975. "Inflation Expectations," Economica, London School of Economics and Political Science, vol. 42(166), pages 123-138, May.
    34. Lee, Kevin C, 1994. "Formation of Price and Cost Inflation Expectations in British Manufacturing Industries: A Multi-Sectoral Analysis," Economic Journal, Royal Economic Society, vol. 104(423), pages 372-385, March.
    35. Kauppi, Eija & Lassila, Jukka & Terasvirta, Timo, 1996. "Short-term forecasting of industrial production with business survey data: experience from Finland's great depression 1990-1993," International Journal of Forecasting, Elsevier, vol. 12(3), pages 373-381, September.
    36. Wheeler, Tracy, 2010. "What can we learn from surveys of business expectations?," Bank of England Quarterly Bulletin, Bank of England, vol. 50(3), pages 190-198.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Oscar Claveria & Enric Monte & Salvador Torra, 2015. "“Self-organizing map analysis of agents' expectations. Different patterns of anticipation of the 2008 financial crisis”," IREA Working Papers 201511, University of Barcelona, Research Institute of Applied Economics, revised Mar 2015.
    2. Breitung, Jörg & Schmeling, Maik, 2013. "Quantifying survey expectations: What’s wrong with the probability approach?," International Journal of Forecasting, Elsevier, vol. 29(1), pages 142-154.
    3. Oscar Claveria & Enric Monte & Salvador Torra, 2018. "“Tracking economic growth by evolving expectations via genetic programming: A two-step approach”," IREA Working Papers 201801, University of Barcelona, Research Institute of Applied Economics, revised Jan 2018.
    4. Puah, Chin-Hong & Wong, Shirly Siew-Ling & Habibullah, Muzafar Shah, 2012. "Rationality of business operational forecasts: evidence from Malaysian distributive trade sector," MPRA Paper 37599, University Library of Munich, Germany.
    5. Lahiri, Kajal & Zhao, Yongchen, 2015. "Quantifying survey expectations: A critical review and generalization of the Carlson–Parkin method," International Journal of Forecasting, Elsevier, vol. 31(1), pages 51-62.
    6. repec:eee:touman:v:47:y:2015:i:c:p:213-223 is not listed on IDEAS
    7. Martinsen, Kjetil & Ravazzolo, Francesco & Wulfsberg, Fredrik, 2014. "Forecasting macroeconomic variables using disaggregate survey data," International Journal of Forecasting, Elsevier, vol. 30(1), pages 65-77.
    8. Kajal Lahiri & Yongchen Zhao, 2013. "Quantifying Heterogeneous Survey Expectations: The Carlson-Parkin Method Revisited," Discussion Papers 13-08, University at Albany, SUNY, Department of Economics.
    9. Oscar Claveria & Enric Monte & Salvador Torra, 2017. "“Let the data do the talking: Empirical modelling of survey-based expectations by means of genetic programming”," AQR Working Papers 201706, University of Barcelona, Regional Quantitative Analysis Group, revised May 2017.
    10. repec:spr:soinre:v:135:y:2018:i:1:d:10.1007_s11205-016-1490-3 is not listed on IDEAS

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:27:y:2011:i:4:p:1128-1146. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Dana Niculescu). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.