IDEAS home Printed from https://ideas.repec.org/p/nsr/niesrd/343.html
   My bibliography  Save this paper

The utility of expectational data: Firm-level evidence using matched qualitative-quantitative UK surveys

Author

Listed:
  • Dr Silvia Lui
  • Dr Martin Weale
  • Dr. James Mitchell

Abstract

This paper assesses the utility of qualitative expectational survey data at the firm-level in terms of both their ability to anticipate firms' subsequent retrospective, but qualitative, reports of their performance but also these same firms' quantitative answers. The assessment requires access to a unique panel dataset which matches firms' responses to a leading qualitative tendency survey conducted by the Confederation of British Industry with these same firms' quantitative replies to a different survey carried out by the Office for National Statistics. We employ nonparametric tests of the so-called 'best-case scenario' and introduce a weaker test for the coherence between these two surveys and test whether the qualitative data contain a(ny) signal about the quantitative data. We find that while firms' qualitative expectations are 'best-case' predictions of their qualitative assessment of their output growth they do not contain a signal about the quantitative data. But we can reject the null hypothesis of noise for the retrospective qualitative data. We discuss this apparent paradox and suggest that qualitative business survey data are more useful for nowcasting than forecasting.

Suggested Citation

  • Dr Silvia Lui & Dr Martin Weale & Dr. James Mitchell, 2009. "The utility of expectational data: Firm-level evidence using matched qualitative-quantitative UK surveys," National Institute of Economic and Social Research (NIESR) Discussion Papers 343, National Institute of Economic and Social Research.
  • Handle: RePEc:nsr:niesrd:343
    as

    Download full text from publisher

    File URL: https://www.niesr.ac.uk/wp-content/uploads/2021/10/dp343_0-2.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Clements, Michael P., 2010. "Explanations of the inconsistencies in survey respondents' forecasts," European Economic Review, Elsevier, vol. 54(4), pages 536-549, May.
    2. Pesaran, M. Hashem & Weale, Martin, 2006. "Survey Expectations," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 14, pages 715-776, Elsevier.
    3. Carroll, Christopher D & Fuhrer, Jeffrey C & Wilcox, David W, 1994. "Does Consumer Sentiment Forecast Household Spending? If So, Why?," American Economic Review, American Economic Association, vol. 84(5), pages 1397-1408, December.
    4. Forni, Mario, et al, 2001. "Coincident and Leading Indicators for the Euro Area," Economic Journal, Royal Economic Society, vol. 111(471), pages 62-85, May.
    5. William A. Branch, 2004. "The Theory of Rationally Heterogeneous Expectations: Evidence from Survey Data on Inflation Expectations," Economic Journal, Royal Economic Society, vol. 114(497), pages 592-621, July.
    6. Keane, Michael P & Runkle, David E, 1990. "Testing the Rationality of Price Forecasts: New Evidence from Panel Data," American Economic Review, American Economic Association, vol. 80(4), pages 714-735, September.
    7. Claveria, Oscar & Pons, Ernest & Ramos, Raul, 2007. "Business and consumer expectations and macroeconomic forecasts," International Journal of Forecasting, Elsevier, vol. 23(1), pages 47-69.
    8. Brown, Bryan W & Maital, Shlomo, 1981. "What Do Economists Know? An Empirical Study of Experts' Expectations," Econometrica, Econometric Society, vol. 49(2), pages 491-504, March.
    9. Troy D. Matheson & James Mitchell & Brian Silverstone, 2010. "Nowcasting and predicting data revisions using panel survey data," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 29(3), pages 313-330.
    10. Easaw, Joshy Z. & Heravi, Saeed M., 2004. "Evaluating consumer sentiments as predictors of UK household consumption behavior: Are they accurate and useful?," International Journal of Forecasting, Elsevier, vol. 20(4), pages 671-681.
    11. Patton, Andrew J. & Timmermann, Allan, 2007. "Properties of optimal forecasts under asymmetric loss and nonlinearity," Journal of Econometrics, Elsevier, vol. 140(2), pages 884-918, October.
    12. Souleles, Nicholas S, 2004. "Expectations, Heterogeneous Forecast Errors, and Consumption: Micro Evidence from the Michigan Consumer Sentiment Surveys," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 36(1), pages 39-72, February.
    13. James Mitchell & Richard J. Smith & Martin R. Weale, 2005. "Forecasting Manufacturing Output Growth Using Firm‐Level Survey Data," Manchester School, University of Manchester, vol. 73(4), pages 479-499, July.
    14. Abberger, Klaus, 2007. "Qualitative business surveys and the assessment of employment -- A case study for Germany," International Journal of Forecasting, Elsevier, vol. 23(2), pages 249-258.
    15. Gourieroux, Christian & Pradel, Jacqueline, 1986. "Direct test of the rational expectation hypothesis," European Economic Review, Elsevier, vol. 30(2), pages 265-284, April.
    16. Stephen G. Hall & James Mitchell, 2009. "Recent Developments in Density Forecasting," Palgrave Macmillan Books, in: Terence C. Mills & Kerry Patterson (ed.), Palgrave Handbook of Econometrics, chapter 5, pages 199-239, Palgrave Macmillan.
    17. Clements,Michael & Hendry,David, 1998. "Forecasting Economic Time Series," Cambridge Books, Cambridge University Press, number 9780521634809.
    18. Das, J.W.M. & Dominitz, J. & van Soest, A.H.O., 1997. "Comparing Predictions and Outcomes : Theory and Application to Income Changes," Other publications TiSEM 6eef11dd-0ae4-4673-b8c0-2, Tilburg University, School of Economics and Management.
    19. Sendhil Mullainathan & Marianne Bertrand, 2001. "Do People Mean What They Say? Implications for Subjective Survey Data," American Economic Review, American Economic Association, vol. 91(2), pages 67-72, May.
    20. Bonham, Carl S & Cohen, Richard H, 2001. "To Aggregate, Pool, or Neither: Testing the Rational-Expectations Hypothesis Using Survey Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(3), pages 278-291, July.
    21. Hansson, Jesper & Jansson, Per & Lof, Marten, 2005. "Business survey data: Do they help in forecasting GDP growth?," International Journal of Forecasting, Elsevier, vol. 21(2), pages 377-389.
    22. Ivaldi, Marc, 1992. "Survey Evidence on the Rationality of Expectations," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 7(3), pages 225-241, July-Sept.
    23. Engelberg, Joseph & Manski, Charles F. & Williams, Jared, 2009. "Comparing the Point Predictions and Subjective Probability Distributions of Professional Forecasters," Journal of Business & Economic Statistics, American Statistical Association, vol. 27, pages 30-41.
    24. Ciaran Driver & Giovanni Urga, 2004. "Transforming Qualitative Survey Data: Performance Comparisons for the UK," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 66(1), pages 71-89, February.
    25. Wallis, Kenneth F, 1980. "Econometric Implications of the Rational Expectations Hypothesis," Econometrica, Econometric Society, vol. 48(1), pages 49-73, January.
    26. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    27. Kauppi, Eija & Lassila, Jukka & Teräsvirta, Timo, 1996. "Short-Term Forecasting of Industrial Production with Business Survey Data: Experience from Finland's Great Depression," Discussion Papers 546, The Research Institute of the Finnish Economy.
    28. Carl S Bonham & Richard H Cohen, 2000. "Testing the Rational Expectations Hypothesis using Survey Data," Working Papers 200007, University of Hawaii at Manoa, Department of Economics.
    29. Patton, Andrew J. & Timmermann, Allan, 2007. "Testing Forecast Optimality Under Unknown Loss," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 1172-1184, December.
    30. Unknown, 1986. "Letters," Choices: The Magazine of Food, Farm, and Resource Issues, Agricultural and Applied Economics Association, vol. 1(4), pages 1-9.
    31. Gourieroux, Christian & Monfort, Alain & Renault, Eric & Trognon, Alain, 1987. "Generalised residuals," Journal of Econometrics, Elsevier, vol. 34(1-2), pages 5-32.
    32. Zellner, Arnold, 1986. "Biased predictors, rationality and the evaluation of forecasts," Economics Letters, Elsevier, vol. 21(1), pages 45-48.
    33. Stock, James H & Watson, Mark W, 2002. "Macroeconomic Forecasting Using Diffusion Indexes," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 147-162, April.
    34. Nerlove, Marc, 1983. "Expectations, Plans, and Realizations in Theory and Practice," Econometrica, Econometric Society, vol. 51(5), pages 1251-1279, September.
    35. James Mitchell & Richard J. Smith & Martin R. Weale, 2005. "Forecasting Manufacturing Output Growth Using Firm‐Level Survey Data," Manchester School, University of Manchester, vol. 73(4), pages 479-499, July.
    36. McIntosh, James & Schiantarelli, Fabio & Low, William, 1989. "A Qualitative Response Analysis of UK Firms' Employment and Output Decisions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 4(3), pages 251-264, July-Sept.
    37. Carlson, John A & Parkin, J Michael, 1975. "Inflation Expectations," Economica, London School of Economics and Political Science, vol. 42(166), pages 123-138, May.
    38. Lee, Kevin C, 1994. "Formation of Price and Cost Inflation Expectations in British Manufacturing Industries: A Multi-Sectoral Analysis," Economic Journal, Royal Economic Society, vol. 104(423), pages 372-385, March.
    39. Wheeler, Tracy, 2010. "What can we learn from surveys of business expectations?," Bank of England Quarterly Bulletin, Bank of England, vol. 50(3), pages 190-198.
    40. Dr Martin Weale & Dr. James Mitchell, 2005. "Forecasting manufacturing output growth using firm-level survey data," National Institute of Economic and Social Research (NIESR) Discussion Papers 251, National Institute of Economic and Social Research.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Boneva, Lena & Cloyne, James & Weale, Martin & Wieladek, Tomasz, 2018. "Firms' Expectations of New Orders, Employment, Costs and Prices: Evidence from Micro Data," CEPR Discussion Papers 12722, C.E.P.R. Discussion Papers.
    2. Puah, Chin-Hong & Wong, Shirly Siew-Ling & Habibullah, Muzafar Shah, 2012. "Rationality of business operational forecasts: evidence from Malaysian distributive trade sector," MPRA Paper 37599, University Library of Munich, Germany.
    3. Lahiri, Kajal & Zhao, Yongchen, 2015. "Quantifying survey expectations: A critical review and generalization of the Carlson–Parkin method," International Journal of Forecasting, Elsevier, vol. 31(1), pages 51-62.
    4. Kevin Lee & Michael Mahony & Paul Mizen, 2020. "The CBI Suite of Business Surveys," Economic Statistics Centre of Excellence (ESCoE) Technical Reports ESCOE-TR-08, Economic Statistics Centre of Excellence (ESCoE).
    5. Lena Boneva & James Cloyne & Martin Weale & Tomasz Wieladek, 2020. "Firms' Price, Cost and Activity Expectations: Evidence from Micro Data," The Economic Journal, Royal Economic Society, vol. 130(627), pages 555-586.
    6. Oscar Claveria & Enric Monte & Salvador Torra, 2018. "Tracking economic growth by evolving expectations via genetic programming: A two-step approach," Working Papers XREAP2018-4, Xarxa de Referència en Economia Aplicada (XREAP), revised Oct 2018.
    7. Oscar Claveria & Enric Monte & Salvador Torra, 2018. "A Data-Driven Approach to Construct Survey-Based Indicators by Means of Evolutionary Algorithms," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 135(1), pages 1-14, January.
    8. Das, Abhiman & Lahiri, Kajal & Zhao, Yongchen, 2019. "Inflation expectations in India: Learning from household tendency surveys," International Journal of Forecasting, Elsevier, vol. 35(3), pages 980-993.
    9. Oscar Claveria & Enric Monte & Salvador Torra, 2017. "Let the data do the talking: Empirical modelling of survey-based expectations by means of genetic programming," IREA Working Papers 201711, University of Barcelona, Research Institute of Applied Economics, revised May 2017.
    10. Oscar Claveria & Enric Monte & Salvador Torra, 2019. "Evolutionary Computation for Macroeconomic Forecasting," Computational Economics, Springer;Society for Computational Economics, vol. 53(2), pages 833-849, February.
    11. Antonecchia, Gianluca, 2023. "Heterogeneous expectations, forecast accuracy and firms’ credit demand," European Economic Review, Elsevier, vol. 154(C).
    12. Oscar Claveria & Enric Monte & Salvador Torra, 2015. "“Self-organizing map analysis of agents' expectations. Different patterns of anticipation of the 2008 financial crisis”," IREA Working Papers 201511, University of Barcelona, Research Institute of Applied Economics, revised Mar 2015.
    13. Breitung, Jörg & Schmeling, Maik, 2013. "Quantifying survey expectations: What’s wrong with the probability approach?," International Journal of Forecasting, Elsevier, vol. 29(1), pages 142-154.
    14. Ferrando, Annalisa & Ganoulis, Ioannis & Preuss, Carsten, 2019. "Firms’ expectations on the availability of credit since the financial crisis," Working Paper Series 2341, European Central Bank.
    15. Guizzardi, Andrea & Stacchini, Annalisa, 2015. "Real-time forecasting regional tourism with business sentiment surveys," Tourism Management, Elsevier, vol. 47(C), pages 213-223.
    16. Martinsen, Kjetil & Ravazzolo, Francesco & Wulfsberg, Fredrik, 2014. "Forecasting macroeconomic variables using disaggregate survey data," International Journal of Forecasting, Elsevier, vol. 30(1), pages 65-77.
    17. Alex Botsis & Kevin Lee, 2022. "Nowcasting Using Firm-Level Survey Data; Tracking UK Output Fluctuations and Recessionary Events," Economic Statistics Centre of Excellence (ESCoE) Technical Reports ESCOE-TR-20, Economic Statistics Centre of Excellence (ESCoE).
    18. Kajal Lahiri & Yongchen Zhao, 2013. "Quantifying Heterogeneous Survey Expectations: The Carlson-Parkin Method Revisited," Discussion Papers 13-08, University at Albany, SUNY, Department of Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lui, Silvia & Mitchell, James & Weale, Martin, 2011. "The utility of expectational data: Firm-level evidence using matched qualitative-quantitative UK surveys," International Journal of Forecasting, Elsevier, vol. 27(4), pages 1128-1146, October.
    2. Oscar Claveria & Enric Monte & Salvador Torra, 2018. "A Data-Driven Approach to Construct Survey-Based Indicators by Means of Evolutionary Algorithms," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 135(1), pages 1-14, January.
    3. Oscar Claveria & Enric Monte & Salvador Torra, 2019. "Evolutionary Computation for Macroeconomic Forecasting," Computational Economics, Springer;Society for Computational Economics, vol. 53(2), pages 833-849, February.
    4. Oscar Claveria & Enric Monte & Salvador Torra, 2018. "Tracking economic growth by evolving expectations via genetic programming: A two-step approach," Working Papers XREAP2018-4, Xarxa de Referència en Economia Aplicada (XREAP), revised Oct 2018.
    5. Dr Martin Weale & Dr. James Mitchell, 2006. "A Bayesian Indicator of Manufacturing Output from Qualitative Business Panel Survey Data," National Institute of Economic and Social Research (NIESR) Discussion Papers 261, National Institute of Economic and Social Research.
    6. Oscar Claveria & Enric Monte & Salvador Torra, 2017. "Let the data do the talking: Empirical modelling of survey-based expectations by means of genetic programming," IREA Working Papers 201711, University of Barcelona, Research Institute of Applied Economics, revised May 2017.
    7. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    8. James Mitchell & Richard J. Smith & Martin R. Weale, 2013. "Efficient Aggregation Of Panel Qualitative Survey Data," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 28(4), pages 580-603, June.
    9. Breitung, Jörg & Schmeling, Maik, 2013. "Quantifying survey expectations: What’s wrong with the probability approach?," International Journal of Forecasting, Elsevier, vol. 29(1), pages 142-154.
    10. Carlos Capistr¡N & Allan Timmermann, 2009. "Disagreement and Biases in Inflation Expectations," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 41(2-3), pages 365-396, March.
    11. Clements, Michael P., 2010. "Explanations of the inconsistencies in survey respondents' forecasts," European Economic Review, Elsevier, vol. 54(4), pages 536-549, May.
    12. Olivier Biau & Hélène Erkel-Rousse & Nicolas Ferrari, 2006. "Réponses individuelles aux enquêtes de conjoncture et prévision de la production manufacturière," Économie et Statistique, Programme National Persée, vol. 395(1), pages 91-116.
    13. Lahiri, Kajal & Zhao, Yongchen, 2015. "Quantifying survey expectations: A critical review and generalization of the Carlson–Parkin method," International Journal of Forecasting, Elsevier, vol. 31(1), pages 51-62.
    14. Oscar Claveria & Enric Monte & Salvador Torra, 2019. "Empirical modelling of survey-based expectations for the design of economic indicators in five European regions," Empirica, Springer;Austrian Institute for Economic Research;Austrian Economic Association, vol. 46(2), pages 205-227, May.
    15. Silvia Lui & James Mitchell & Martin Weale, 2011. "Qualitative business surveys: signal or noise?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(2), pages 327-348, April.
    16. Kajal Lahiri & Yongchen Zhao, 2013. "Quantifying Heterogeneous Survey Expectations: The Carlson-Parkin Method Revisited," Discussion Papers 13-08, University at Albany, SUNY, Department of Economics.
    17. Michael P. Clements, 2014. "US Inflation Expectations and Heterogeneous Loss Functions, 1968–2010," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 33(1), pages 1-14, January.
    18. Pesaran, M. Hashem & Weale, Martin, 2006. "Survey Expectations," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 14, pages 715-776, Elsevier.
    19. Mitchell, James & Weale, Martin R., 2007. "The rationality and reliability of expectations reported by British households: micro evidence from the British household panel survey," Discussion Paper Series 1: Economic Studies 2007,19, Deutsche Bundesbank.
    20. Ivana Komunjer & Michael T. Owyang, 2012. "Multivariate Forecast Evaluation and Rationality Testing," The Review of Economics and Statistics, MIT Press, vol. 94(4), pages 1066-1080, November.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nsr:niesrd:343. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Library & Information Manager (email available below). General contact details of provider: https://edirc.repec.org/data/niesruk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.