IDEAS home Printed from https://ideas.repec.org/p/nsr/escoed/escoe-dp-2021-14.html
   My bibliography  Save this paper

Quantifying Qualitative Survey Data: New Insights on the (Ir)Rationality of Firms' Forecasts

Author

Listed:
  • Alex Botsis
  • Christoph Gortz
  • Plutarchos Sakellaris

Abstract

Using a novel dataset that combines firms' qualitative survey-based sales forecasts with their quantitative balance-sheet data on realized sales, we document that only major forecast errors (those in the two tails of the distribution) are predictable and display autocorrelation. This result is a particular violation of the Full Information Rational Expectations hypothesis that requires explanation. In contrast, minor forecast errors are neither predictable nor autocorrelated. To arrive at this finding, we develop a novel methodology to quantify qualitative survey data on forecasts. It is generally applicable when quantitative information, e.g. from firm balance sheets, is available on the realization of the forecasted variable. To explain our empirical result, we provide a model of rational inattention. When operating in market environments where information processing is more costly, firms optimally limit their degree of attention to information. This results in larger absolute forecast errors that become predictable and autocorrelated.

Suggested Citation

  • Alex Botsis & Christoph Gortz & Plutarchos Sakellaris, 2021. "Quantifying Qualitative Survey Data: New Insights on the (Ir)Rationality of Firms' Forecasts," Economic Statistics Centre of Excellence (ESCoE) Discussion Papers ESCoE DP-2021-14, Economic Statistics Centre of Excellence (ESCoE).
  • Handle: RePEc:nsr:escoed:escoe-dp-2021-14
    as

    Download full text from publisher

    File URL: https://escoe-website.s3.amazonaws.com/wp-content/uploads/2021/10/28110518/ESCoE-DP-2021-14-.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Silvia Lui & James Mitchell & Martin Weale, 2011. "Qualitative business surveys: signal or noise?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(2), pages 327-348, April.
    2. Enders, Zeno & Hünnekes, Franziska & Müller, Gernot J., 2019. "Monetary policy announcements and expectations: Evidence from german firms," Journal of Monetary Economics, Elsevier, vol. 108(C), pages 45-63.
    3. Olivier Coibion & Yuriy Gorodnichenko & Saten Kumar, 2018. "How Do Firms Form Their Expectations? New Survey Evidence," American Economic Review, American Economic Association, vol. 108(9), pages 2671-2713, September.
    4. Zeno Enders & Franziska Hünnekes & Gernot Müller, 2022. "Firm Expectations and Economic Activity," Journal of the European Economic Association, European Economic Association, vol. 20(6), pages 2396-2439.
    5. Pedro Bordalo & Nicola Gennaioli & Yueran Ma & Andrei Shleifer, 2020. "Overreaction in Macroeconomic Expectations," American Economic Review, American Economic Association, vol. 110(9), pages 2748-2782, September.
    6. Arellano, Manuel & Bover, Olympia, 1995. "Another look at the instrumental variable estimation of error-components models," Journal of Econometrics, Elsevier, vol. 68(1), pages 29-51, July.
    7. Nicholas Bloom & Philip Bunn & Scarlet Chen & Paul Mizen & Pawel Smietanka & Gregory Thwaites, 2019. "The Impact of Brexit on UK Firms," NBER Working Papers 26218, National Bureau of Economic Research, Inc.
    8. Olivier Coibion & Yuriy Gorodnichenko, 2012. "What Can Survey Forecasts Tell Us about Information Rigidities?," Journal of Political Economy, University of Chicago Press, vol. 120(1), pages 116-159.
    9. Nicola Gennaioli & Yueran Ma & Andrei Shleifer, 2016. "Expectations and Investment," NBER Macroeconomics Annual, University of Chicago Press, vol. 30(1), pages 379-431.
    10. Smith, Jeremy & McAleer, Michael, 1995. "Alternative Procedures for Converting Qualitative Response Data to Quantitative Expectations: An Application to Australian Manufacturing," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 10(2), pages 165-185, April-Jun.
    11. Olga Gorbachev, 2011. "Did Household Consumption Become More Volatile?," American Economic Review, American Economic Association, vol. 101(5), pages 2248-2270, August.
    12. Xavier Gabaix, 2014. "A Sparsity-Based Model of Bounded Rationality," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 129(4), pages 1661-1710.
    13. Triguero, Ángela & Córcoles, David, 2013. "Understanding innovation: An analysis of persistence for Spanish manufacturing firms," Research Policy, Elsevier, vol. 42(2), pages 340-352.
    14. Carlos Capistr¡N & Allan Timmermann, 2009. "Disagreement and Biases in Inflation Expectations," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 41(2-3), pages 365-396, March.
    15. N. Gregory Mankiw & Ricardo Reis, 2002. "Sticky Information versus Sticky Prices: A Proposal to Replace the New Keynesian Phillips Curve," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 117(4), pages 1295-1328.
    16. Bartosz Mackowiak & Mirko Wiederholt, 2009. "Optimal Sticky Prices under Rational Inattention," American Economic Review, American Economic Association, vol. 99(3), pages 769-803, June.
    17. Tanaka, Mari & Bloom, Nicholas & David, Joel M. & Koga, Maiko, 2020. "Firm performance and macro forecast accuracy," Journal of Monetary Economics, Elsevier, vol. 114(C), pages 26-41.
    18. Massenot, Baptiste & Pettinicchi, Yuri, 2018. "Can firms see into the future? Survey evidence from Germany," Journal of Economic Behavior & Organization, Elsevier, vol. 145(C), pages 66-79.
    19. Hajdini, Ina & Kurmann, Andre, 2022. "Predictable Forecast Errors in Full-Information Rational Expectations Models with Regime Shifts," School of Economics Working Paper Series 2022-5, LeBow College of Business, Drexel University.
    20. Michael L. Polemis & Thanasis Stengos, 2019. "Does competition prevent industrial pollution? Evidence from a panel threshold model," Business Strategy and the Environment, Wiley Blackwell, vol. 28(1), pages 98-110, January.
    21. Francesco Caselli & Andrea Tesei, 2016. "Resource Windfalls, Political Regimes, and Political Stability," The Review of Economics and Statistics, MIT Press, vol. 98(3), pages 573-590, July.
    22. Massenot, Baptiste & Pettinicchi, Yuri, 2019. "Can households see into the future? Survey evidence from the Netherlands," Journal of Economic Behavior & Organization, Elsevier, vol. 164(C), pages 77-90.
    23. Leland Farmer & Emi Nakamura & Jón Steinsson, 2021. "Learning About the Long Run," NBER Working Papers 29495, National Bureau of Economic Research, Inc.
    24. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    25. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, April.
    26. Nicholas Bloom & Takafumi Kawakubo & Charlotte Chunming Meng & Paul Mizen & Rebecca Riley & Tatsuro Senga & John Van Reenen, 2021. "Do Well Managed Firms Make Better Forecasts?," Economic Statistics Centre of Excellence (ESCoE) Discussion Papers ESCoE DP-2021-18, Economic Statistics Centre of Excellence (ESCoE).
    27. Charles F. Manski, 2004. "Measuring Expectations," Econometrica, Econometric Society, vol. 72(5), pages 1329-1376, September.
    28. Bachmann, Rüdiger & Zorn, Peter, 2020. "What drives aggregate investment? Evidence from German survey data," Journal of Economic Dynamics and Control, Elsevier, vol. 115(C).
    29. Stock, James H & Wright, Jonathan H & Yogo, Motohiro, 2002. "A Survey of Weak Instruments and Weak Identification in Generalized Method of Moments," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(4), pages 518-529, October.
    30. Olivier Coibion & Yuriy Gorodnichenko, 2015. "Information Rigidity and the Expectations Formation Process: A Simple Framework and New Facts," American Economic Review, American Economic Association, vol. 105(8), pages 2644-2678, August.
    31. Bachmann, Rüdiger & Elstner, Steffen, 2015. "Firm optimism and pessimism," European Economic Review, Elsevier, vol. 79(C), pages 297-325.
    32. Pedro Bordalo & Nicola Gennaioli & Andrei Shleifer, 2018. "Diagnostic Expectations and Credit Cycles," Journal of Finance, American Finance Association, vol. 73(1), pages 199-227, February.
    33. David Roodman, 2009. "A Note on the Theme of Too Many Instruments," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 71(1), pages 135-158, February.
    34. Semykina, Anastasia & Wooldridge, Jeffrey M., 2010. "Estimating panel data models in the presence of endogeneity and selection," Journal of Econometrics, Elsevier, vol. 157(2), pages 375-380, August.
    35. Renáta Kosová, 2010. "Do Foreign Firms Crowd Out Domestic Firms? Evidence from the Czech Republic," The Review of Economics and Statistics, MIT Press, vol. 92(4), pages 861-881, November.
    36. Matějka, Filip & Mackowiak, Bartosz & Wiederholt, Mirko, 2018. "Survey: Rational Inattention, a Disciplined Behavioral Model," CEPR Discussion Papers 13243, C.E.P.R. Discussion Papers.
    37. Nathanael Vellekoop & Mirko Wiederholt, 2019. "Inflation Expectations and Choices of Households," SciencePo Working papers Main hal-03878694, HAL.
    38. Bartosz Maćkowiak & Filip Matějka & Mirko Wiederholt, 2023. "Rational Inattention: A Review," Journal of Economic Literature, American Economic Association, vol. 61(1), pages 226-273, March.
    39. Seo, Myung Hwan & Shin, Yongcheol, 2016. "Dynamic panels with threshold effect and endogeneity," Journal of Econometrics, Elsevier, vol. 195(2), pages 169-186.
    40. Pesaran, M. Hashem, 2015. "Time Series and Panel Data Econometrics," OUP Catalogue, Oxford University Press, number 9780198759980.
    41. Frieder Mokinski & Xuguang (Simon) Sheng & Jingyun Yang, 2015. "Measuring Disagreement in Qualitative Expectations," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 34(5), pages 405-426, August.
    42. R?diger Bachmann & Steffen Elstner & Eric R. Sims, 2013. "Uncertainty and Economic Activity: Evidence from Business Survey Data," American Economic Journal: Macroeconomics, American Economic Association, vol. 5(2), pages 217-249, April.
    43. Andrew Caplin & Mark Dean, 2015. "Revealed Preference, Rational Inattention, and Costly Information Acquisition," American Economic Review, American Economic Association, vol. 105(7), pages 2183-2203, July.
    44. Asimakopoulos, Stylianos & Karavias, Yiannis, 2016. "The impact of government size on economic growth: A threshold analysis," Economics Letters, Elsevier, vol. 139(C), pages 65-68.
    45. Sushant Acharya & Shu Lin Wee, 2020. "Rational Inattention in Hiring Decisions," American Economic Journal: Macroeconomics, American Economic Association, vol. 12(1), pages 1-40, January.
    46. Carlos Capistrán & Allan Timmermann, 2009. "Disagreement and Biases in Inflation Expectations," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 41(2‐3), pages 365-396, March.
    47. Mundlak, Yair, 1978. "On the Pooling of Time Series and Cross Section Data," Econometrica, Econometric Society, vol. 46(1), pages 69-85, January.
    48. Nickell, Stephen J, 1981. "Biases in Dynamic Models with Fixed Effects," Econometrica, Econometric Society, vol. 49(6), pages 1417-1426, November.
    49. Nerlove, Marc, 1983. "Expectations, Plans, and Realizations in Theory and Practice," Econometrica, Econometric Society, vol. 51(5), pages 1251-1279, September.
    50. Sims, Christopher A., 2003. "Implications of rational inattention," Journal of Monetary Economics, Elsevier, vol. 50(3), pages 665-690, April.
    51. Bartelsman, Eric J & Caballero, Ricardo J & Lyons, Richard K, 1994. "Customer- and Supplier-Driven Externalities," American Economic Review, American Economic Association, vol. 84(4), pages 1075-1084, September.
    52. Manuel Arellano & Stephen Bond, 1991. "Some Tests of Specification for Panel Data: Monte Carlo Evidence and an Application to Employment Equations," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 58(2), pages 277-297.
    53. Windmeijer, Frank, 2005. "A finite sample correction for the variance of linear efficient two-step GMM estimators," Journal of Econometrics, Elsevier, vol. 126(1), pages 25-51, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Monique B. Reid & Pierre L. Siklos, 2022. "How Firms and Experts View The Phillips Curve: Evidence from Individual and Aggregate Data from South Africa," Emerging Markets Finance and Trade, Taylor & Francis Journals, vol. 58(12), pages 3355-3376, September.
    2. Alex Botsis & Kevin Lee, 2022. "Nowcasting Using Firm-Level Survey Data; Tracking UK Output Fluctuations and Recessionary Events," Economic Statistics Centre of Excellence (ESCoE) Technical Reports ESCOE-TR-20, Economic Statistics Centre of Excellence (ESCoE).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alexandros Botsis & Christoph Görtz & Plutarchos Sakellaris, 2024. "Quantifying Qualitative Survey Data with Panel Data Structure," CESifo Working Paper Series 11013, CESifo.
    2. Chen, Cheng & Senga, Tatsuro & Sun, Chang & Zhang, Hongyong, 2023. "Uncertainty, imperfect information, and expectation formation over the firm’s life cycle," Journal of Monetary Economics, Elsevier, vol. 140(C), pages 60-77.
    3. Born, Benjamin & Enders, Zeno & Menkhoff, Manuel & Müller, Gernot & Niemann, Knut, 2022. "Firm Expectations and News: Micro v Macro," CEPR Discussion Papers 17768, C.E.P.R. Discussion Papers.
    4. Massenot, Baptiste & Pettinicchi, Yuri, 2018. "Can firms see into the future? Survey evidence from Germany," Journal of Economic Behavior & Organization, Elsevier, vol. 145(C), pages 66-79.
    5. Zeno Enders & Franziska Hünnekes & Gernot Müller, 2022. "Firm Expectations and Economic Activity," Journal of the European Economic Association, European Economic Association, vol. 20(6), pages 2396-2439.
    6. Andrade, Philippe & Gautier, Erwan & Mengus, Eric, 2023. "What matters in households’ inflation expectations?," Journal of Monetary Economics, Elsevier, vol. 138(C), pages 50-68.
    7. Antonecchia, Gianluca, 2023. "Heterogeneous expectations, forecast accuracy and firms’ credit demand," European Economic Review, Elsevier, vol. 154(C).
    8. Bartosz Maćkowiak & Filip Matějka & Mirko Wiederholt, 2023. "Rational Inattention: A Review," Journal of Economic Literature, American Economic Association, vol. 61(1), pages 226-273, March.
    9. Frache, Serafin & Lluberas, Rodrigo & Turen, Javier, 2024. "Belief-dependent pricing decisions," Economic Modelling, Elsevier, vol. 132(C).
    10. Enders, Zeno & Hünnekes, Franziska & Müller, Gernot J., 2019. "Monetary policy announcements and expectations: Evidence from german firms," Journal of Monetary Economics, Elsevier, vol. 108(C), pages 45-63.
    11. Dovern, Jonas & Müller, Lena Sophia & Wohlrabe, Klaus, 2023. "Local information and firm expectations about aggregates," Journal of Monetary Economics, Elsevier, vol. 138(C), pages 1-13.
    12. Lukas Buchheim & Sebastian Link, 2017. "The Effect of Disaggregate Information on the Expectation Formation of Firms," CESifo Working Paper Series 6768, CESifo.
    13. Altig, David & Barrero, Jose Maria & Bloom, Nicholas & Davis, Steven J. & Meyer, Brent & Parker, Nicholas, 2022. "Surveying business uncertainty," Journal of Econometrics, Elsevier, vol. 231(1), pages 282-303.
    14. Bachmann, Rüdiger, 2019. "Comments on “Monetary policy announcements and expectations: Evidence from German firms”," Journal of Monetary Economics, Elsevier, vol. 108(C), pages 64-68.
    15. Nakazono, Yoshiyuki & Koga, Maiko & Sugo, Tomohiro, 2020. "Private information and analyst coverage: Evidence from firm survey data," Journal of Economic Behavior & Organization, Elsevier, vol. 174(C), pages 284-298.
    16. Link, Sebastian & Peichl, Andreas & Roth, Christopher & Wohlfart, Johannes, 2023. "Information frictions among firms and households," Journal of Monetary Economics, Elsevier, vol. 135(C), pages 99-115.
    17. Olivier Coibion & Yuriy Gorodnichenko & Tiziano Ropele, 2020. "Inflation Expectations and Firm Decisions: New Causal Evidence," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 135(1), pages 165-219.
    18. Dovern, Jonas & Fritsche, Ulrich & Loungani, Prakash & Tamirisa, Natalia, 2015. "Information rigidities: Comparing average and individual forecasts for a large international panel," International Journal of Forecasting, Elsevier, vol. 31(1), pages 144-154.
    19. Angeletos, G.-M. & Lian, C., 2016. "Incomplete Information in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 1065-1240, Elsevier.
    20. Kryvtsov, Oleksiy & Petersen, Luba, 2021. "Central bank communication that works: Lessons from lab experiments," Journal of Monetary Economics, Elsevier, vol. 117(C), pages 760-780.

    More about this item

    Keywords

    expectations; firm data; forecast errors; panel threshold models; rational inattention; survey data;
    All these keywords.

    JEL classification:

    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • D22 - Microeconomics - - Production and Organizations - - - Firm Behavior: Empirical Analysis
    • D84 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Expectations; Speculations
    • E32 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Business Fluctuations; Cycles

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nsr:escoed:escoe-dp-2021-14. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ESCoE Centre Manager (email available below). General contact details of provider: https://edirc.repec.org/data/escoeuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.