IDEAS home Printed from https://ideas.repec.org/p/bir/birmec/23-06.html
   My bibliography  Save this paper

Quantifying Qualitative Survey Data: New Insights on the (Ir)Rationality of Firms' Forecasts

Author

Listed:
  • Alexandros Botsis

    (Economic Statistics Centre of Excellence)

  • Christoph Gortz

    (University of Birmingham)

  • Plutarchos Sakellaris

    (Athens University of Economics and Business)

Abstract

Using a novel dataset that combines firms’ qualitative survey-based sales forecasts with quantitative balance-sheet data on realized sales, we document that only major forecast errors (those in the two distribution tails) are predictable and autocorrelated. This is a particular violation of the Full Information Rational Expectations hypothesis that requires explanation. In contrast, minor forecast errors are neither predictable nor autocorrelated. To arrive at this finding, we develop a novel methodology to quantify qualitative survey data on firm forecasts of own growth. It is generally applicable when quantitative information is available on the realization of the forecasted variable, for example from firm balance sheets. The method can also be applied to panel data on qualitative household expectations.

Suggested Citation

  • Alexandros Botsis & Christoph Gortz & Plutarchos Sakellaris, 2023. "Quantifying Qualitative Survey Data: New Insights on the (Ir)Rationality of Firms' Forecasts," Discussion Papers 23-06, Department of Economics, University of Birmingham.
  • Handle: RePEc:bir:birmec:23-06
    as

    Download full text from publisher

    File URL: https://repec.cal.bham.ac.uk/pdf/23-06.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. R?diger Bachmann & Steffen Elstner & Eric R. Sims, 2013. "Uncertainty and Economic Activity: Evidence from Business Survey Data," American Economic Journal: Macroeconomics, American Economic Association, vol. 5(2), pages 217-249, April.
    2. Silvia Lui & James Mitchell & Martin Weale, 2011. "Qualitative business surveys: signal or noise?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(2), pages 327-348, April.
    3. Seo, Myung Hwan & Shin, Yongcheol, 2016. "Dynamic panels with threshold effect and endogeneity," Journal of Econometrics, Elsevier, vol. 195(2), pages 169-186.
    4. Pesaran, M. Hashem, 2015. "Time Series and Panel Data Econometrics," OUP Catalogue, Oxford University Press, number 9780198759980.
    5. Sushant Acharya & Shu Lin Wee, 2020. "Rational Inattention in Hiring Decisions," American Economic Journal: Macroeconomics, American Economic Association, vol. 12(1), pages 1-40, January.
    6. Bachmann, Rüdiger & Elstner, Steffen, 2015. "Firm optimism and pessimism," European Economic Review, Elsevier, vol. 79(C), pages 297-325.
    7. Enders, Zeno & Hünnekes, Franziska & Müller, Gernot J., 2019. "Monetary policy announcements and expectations: Evidence from german firms," Journal of Monetary Economics, Elsevier, vol. 108(C), pages 45-63.
    8. Frieder Mokinski & Xuguang (Simon) Sheng & Jingyun Yang, 2015. "Measuring Disagreement in Qualitative Expectations," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 34(5), pages 405-426, August.
    9. Massenot, Baptiste & Pettinicchi, Yuri, 2018. "Can firms see into the future? Survey evidence from Germany," Journal of Economic Behavior & Organization, Elsevier, vol. 145(C), pages 66-79.
    10. Olivier Coibion & Yuriy Gorodnichenko & Saten Kumar, 2018. "How Do Firms Form Their Expectations? New Survey Evidence," American Economic Review, American Economic Association, vol. 108(9), pages 2671-2713, September.
    11. Francesco Caselli & Andrea Tesei, 2016. "Resource Windfalls, Political Regimes, and Political Stability," The Review of Economics and Statistics, MIT Press, vol. 98(3), pages 573-590, July.
    12. N. Gregory Mankiw & Ricardo Reis, 2002. "Sticky Information versus Sticky Prices: A Proposal to Replace the New Keynesian Phillips Curve," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 117(4), pages 1295-1328.
    13. Zeno Enders & Franziska Hünnekes & Gernot Müller, 2022. "Firm Expectations and Economic Activity," Journal of the European Economic Association, European Economic Association, vol. 20(6), pages 2396-2439.
    14. Bachmann, Rüdiger & Zorn, Peter, 2020. "What drives aggregate investment? Evidence from German survey data," Journal of Economic Dynamics and Control, Elsevier, vol. 115(C).
    15. Pedro Bordalo & Nicola Gennaioli & Yueran Ma & Andrei Shleifer, 2020. "Overreaction in Macroeconomic Expectations," American Economic Review, American Economic Association, vol. 110(9), pages 2748-2782, September.
    16. Arellano, Manuel & Bover, Olympia, 1995. "Another look at the instrumental variable estimation of error-components models," Journal of Econometrics, Elsevier, vol. 68(1), pages 29-51, July.
    17. Nicholas Bloom & Takafumi Kawakubo & Charlotte Meng & Paul Mizen & Rebecca Riley & Tatsuro Senga & John Van Reenen, 2021. "Do Well Managed Firms Make Better Forecasts?," NBER Working Papers 29591, National Bureau of Economic Research, Inc.
    18. Andrew Caplin & Mark Dean, 2015. "Revealed Preference, Rational Inattention, and Costly Information Acquisition," American Economic Review, American Economic Association, vol. 105(7), pages 2183-2203, July.
    19. Nicholas Bloom & Philip Bunn & Scarlet Chen & Paul Mizen & Pawel Smietanka & Gregory Thwaites, 2019. "The Impact of Brexit on UK Firms," NBER Working Papers 26218, National Bureau of Economic Research, Inc.
    20. Olivier Coibion & Yuriy Gorodnichenko, 2012. "What Can Survey Forecasts Tell Us about Information Rigidities?," Journal of Political Economy, University of Chicago Press, vol. 120(1), pages 116-159.
    21. Nicola Gennaioli & Yueran Ma & Andrei Shleifer, 2016. "Expectations and Investment," NBER Macroeconomics Annual, University of Chicago Press, vol. 30(1), pages 379-431.
    22. Asimakopoulos, Stylianos & Karavias, Yiannis, 2016. "The impact of government size on economic growth: A threshold analysis," Economics Letters, Elsevier, vol. 139(C), pages 65-68.
    23. Sims, Christopher A., 2003. "Implications of rational inattention," Journal of Monetary Economics, Elsevier, vol. 50(3), pages 665-690, April.
    24. Smith, Jeremy & McAleer, Michael, 1995. "Alternative Procedures for Converting Qualitative Response Data to Quantitative Expectations: An Application to Australian Manufacturing," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 10(2), pages 165-185, April-Jun.
    25. Olga Gorbachev, 2011. "Did Household Consumption Become More Volatile?," American Economic Review, American Economic Association, vol. 101(5), pages 2248-2270, August.
    26. Stock, James H & Wright, Jonathan H & Yogo, Motohiro, 2002. "A Survey of Weak Instruments and Weak Identification in Generalized Method of Moments," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(4), pages 518-529, October.
    27. Xavier Gabaix, 2014. "A Sparsity-Based Model of Bounded Rationality," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 129(4), pages 1661-1710.
    28. Triguero, Ángela & Córcoles, David, 2013. "Understanding innovation: An analysis of persistence for Spanish manufacturing firms," Research Policy, Elsevier, vol. 42(2), pages 340-352.
    29. Carlos Capistr¡N & Allan Timmermann, 2009. "Disagreement and Biases in Inflation Expectations," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 41(2-3), pages 365-396, March.
    30. Olivier Coibion & Yuriy Gorodnichenko, 2015. "Information Rigidity and the Expectations Formation Process: A Simple Framework and New Facts," American Economic Review, American Economic Association, vol. 105(8), pages 2644-2678, August.
    31. Bartosz Mackowiak & Mirko Wiederholt, 2009. "Optimal Sticky Prices under Rational Inattention," American Economic Review, American Economic Association, vol. 99(3), pages 769-803, June.
    32. Pedro Bordalo & Nicola Gennaioli & Andrei Shleifer, 2018. "Diagnostic Expectations and Credit Cycles," Journal of Finance, American Finance Association, vol. 73(1), pages 199-227, February.
    33. Nathanael Vellekoop & Mirko Wiederholt, 2019. "Inflation Expectations and Choices of Households," Working Papers hal-03878694, HAL.
    34. David Roodman, 2009. "A Note on the Theme of Too Many Instruments," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 71(1), pages 135-158, February.
    35. Semykina, Anastasia & Wooldridge, Jeffrey M., 2010. "Estimating panel data models in the presence of endogeneity and selection," Journal of Econometrics, Elsevier, vol. 157(2), pages 375-380, August.
    36. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    37. Nerlove, Marc, 1983. "Expectations, Plans, and Realizations in Theory and Practice," Econometrica, Econometric Society, vol. 51(5), pages 1251-1279, September.
    38. Carlos Capistrán & Allan Timmermann, 2009. "Disagreement and Biases in Inflation Expectations," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 41(2‐3), pages 365-396, March.
    39. Tanaka, Mari & Bloom, Nicholas & David, Joel M. & Koga, Maiko, 2020. "Firm performance and macro forecast accuracy," Journal of Monetary Economics, Elsevier, vol. 114(C), pages 26-41.
    40. Renáta Kosová, 2010. "Do Foreign Firms Crowd Out Domestic Firms? Evidence from the Czech Republic," The Review of Economics and Statistics, MIT Press, vol. 92(4), pages 861-881, November.
    41. Hajdini, Ina & Kurmann, Andre, 2022. "Predictable Forecast Errors in Full-Information Rational Expectations Models with Regime Shifts," School of Economics Working Paper Series 2022-5, LeBow College of Business, Drexel University.
    42. Bartelsman, Eric J & Caballero, Ricardo J & Lyons, Richard K, 1994. "Customer- and Supplier-Driven Externalities," American Economic Review, American Economic Association, vol. 84(4), pages 1075-1084, September.
    43. Mundlak, Yair, 1978. "On the Pooling of Time Series and Cross Section Data," Econometrica, Econometric Society, vol. 46(1), pages 69-85, January.
    44. Michael L. Polemis & Thanasis Stengos, 2019. "Does competition prevent industrial pollution? Evidence from a panel threshold model," Business Strategy and the Environment, Wiley Blackwell, vol. 28(1), pages 98-110, January.
    45. Matějka, Filip & Mackowiak, Bartosz & Wiederholt, Mirko, 2018. "Survey: Rational Inattention, a Disciplined Behavioral Model," CEPR Discussion Papers 13243, C.E.P.R. Discussion Papers.
    46. Manuel Arellano & Stephen Bond, 1991. "Some Tests of Specification for Panel Data: Monte Carlo Evidence and an Application to Employment Equations," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 58(2), pages 277-297.
    47. Massenot, Baptiste & Pettinicchi, Yuri, 2019. "Can households see into the future? Survey evidence from the Netherlands," Journal of Economic Behavior & Organization, Elsevier, vol. 164(C), pages 77-90.
    48. Bartosz Maćkowiak & Filip Matějka & Mirko Wiederholt, 2023. "Rational Inattention: A Review," Journal of Economic Literature, American Economic Association, vol. 61(1), pages 226-273, March.
    49. Leland Farmer & Emi Nakamura & Jón Steinsson, 2021. "Learning About the Long Run," NBER Working Papers 29495, National Bureau of Economic Research, Inc.
    50. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, December.
    51. Charles F. Manski, 2004. "Measuring Expectations," Econometrica, Econometric Society, vol. 72(5), pages 1329-1376, September.
    52. Nickell, Stephen J, 1981. "Biases in Dynamic Models with Fixed Effects," Econometrica, Econometric Society, vol. 49(6), pages 1417-1426, November.
    53. Windmeijer, Frank, 2005. "A finite sample correction for the variance of linear efficient two-step GMM estimators," Journal of Econometrics, Elsevier, vol. 126(1), pages 25-51, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Monique B. Reid & Pierre L. Siklos, 2022. "How Firms and Experts View The Phillips Curve: Evidence from Individual and Aggregate Data from South Africa," Emerging Markets Finance and Trade, Taylor & Francis Journals, vol. 58(12), pages 3355-3376, September.
    2. Alex Botsis & Kevin Lee, 2022. "Nowcasting Using Firm-Level Survey Data; Tracking UK Output Fluctuations and Recessionary Events," Economic Statistics Centre of Excellence (ESCoE) Technical Reports ESCOE-TR-20, Economic Statistics Centre of Excellence (ESCoE).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chen, Cheng & Senga, Tatsuro & Sun, Chang & Zhang, Hongyong, 2023. "Uncertainty, imperfect information, and expectation formation over the firm’s life cycle," Journal of Monetary Economics, Elsevier, vol. 140(C), pages 60-77.
    2. Born, Benjamin & Enders, Zeno & Menkhoff, Manuel & Müller, Gernot & Niemann, Knut, 2022. "Firm Expectations and News: Micro v Macro," CEPR Discussion Papers 17768, C.E.P.R. Discussion Papers.
    3. Massenot, Baptiste & Pettinicchi, Yuri, 2018. "Can firms see into the future? Survey evidence from Germany," Journal of Economic Behavior & Organization, Elsevier, vol. 145(C), pages 66-79.
    4. Zeno Enders & Franziska Hünnekes & Gernot Müller, 2022. "Firm Expectations and Economic Activity," Journal of the European Economic Association, European Economic Association, vol. 20(6), pages 2396-2439.
    5. Andrade, Philippe & Gautier, Erwan & Mengus, Eric, 2023. "What matters in households’ inflation expectations?," Journal of Monetary Economics, Elsevier, vol. 138(C), pages 50-68.
    6. Antonecchia, Gianluca, 2023. "Heterogeneous expectations, forecast accuracy and firms’ credit demand," European Economic Review, Elsevier, vol. 154(C).
    7. Bartosz Maćkowiak & Filip Matějka & Mirko Wiederholt, 2023. "Rational Inattention: A Review," Journal of Economic Literature, American Economic Association, vol. 61(1), pages 226-273, March.
    8. Enders, Zeno & Hünnekes, Franziska & Müller, Gernot J., 2019. "Monetary policy announcements and expectations: Evidence from german firms," Journal of Monetary Economics, Elsevier, vol. 108(C), pages 45-63.
    9. Dovern, Jonas & Müller, Lena Sophia & Wohlrabe, Klaus, 2023. "Local information and firm expectations about aggregates," Journal of Monetary Economics, Elsevier, vol. 138(C), pages 1-13.
    10. Lukas Buchheim & Sebastian Link, 2017. "The Effect of Disaggregate Information on the Expectation Formation of Firms," CESifo Working Paper Series 6768, CESifo.
    11. Altig, David & Barrero, Jose Maria & Bloom, Nicholas & Davis, Steven J. & Meyer, Brent & Parker, Nicholas, 2022. "Surveying business uncertainty," Journal of Econometrics, Elsevier, vol. 231(1), pages 282-303.
    12. Bachmann, Rüdiger, 2019. "Comments on “Monetary policy announcements and expectations: Evidence from German firms”," Journal of Monetary Economics, Elsevier, vol. 108(C), pages 64-68.
    13. Nakazono, Yoshiyuki & Koga, Maiko & Sugo, Tomohiro, 2020. "Private information and analyst coverage: Evidence from firm survey data," Journal of Economic Behavior & Organization, Elsevier, vol. 174(C), pages 284-298.
    14. Link, Sebastian & Peichl, Andreas & Roth, Christopher & Wohlfart, Johannes, 2023. "Information frictions among firms and households," Journal of Monetary Economics, Elsevier, vol. 135(C), pages 99-115.
    15. Olivier Coibion & Yuriy Gorodnichenko & Tiziano Ropele, 2020. "Inflation Expectations and Firm Decisions: New Causal Evidence," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 135(1), pages 165-219.
    16. Dovern, Jonas & Fritsche, Ulrich & Loungani, Prakash & Tamirisa, Natalia, 2015. "Information rigidities: Comparing average and individual forecasts for a large international panel," International Journal of Forecasting, Elsevier, vol. 31(1), pages 144-154.
    17. Angeletos, G.-M. & Lian, C., 2016. "Incomplete Information in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 1065-1240, Elsevier.
    18. Kryvtsov, Oleksiy & Petersen, Luba, 2021. "Central bank communication that works: Lessons from lab experiments," Journal of Monetary Economics, Elsevier, vol. 117(C), pages 760-780.
    19. Nathan Goldstein & Ben‐Zion Zilberfarb, 2023. "The closer we get, the better we are?," Economic Inquiry, Western Economic Association International, vol. 61(2), pages 364-376, April.
    20. Anmol Bhandari & Jaroslav Borovicka & Paul Ho, 2019. "Survey Data and Subjective Beliefs in Business Cycle Models," Working Paper 19-14, Federal Reserve Bank of Richmond.

    More about this item

    Keywords

    Expectations; Firm Data; Forecast Errors; Panel Threshold Models; Survey Data.;
    All these keywords.

    JEL classification:

    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • D22 - Microeconomics - - Production and Organizations - - - Firm Behavior: Empirical Analysis
    • D84 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Expectations; Speculations
    • E32 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Business Fluctuations; Cycles

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bir:birmec:23-06. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oleksandr Talavera (email available below). General contact details of provider: https://edirc.repec.org/data/debhauk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.