IDEAS home Printed from https://ideas.repec.org/a/wly/quante/v14y2023i2p609-650.html
   My bibliography  Save this article

Quantifying noise in survey expectations

Author

Listed:
  • Artūras Juodis
  • Simas Kučinskas

Abstract

Expectations affect economic decisions, and inaccurate expectations are costly. Expectations can be wrong due to either bias (systematic mistakes) or noise (unsystematic mistakes). We develop a framework for quantifying the level of noise in survey expectations. The method is based on the insight that theoretical models of expectation formation predict a factor structure for individual expectations. Using data from professional forecasters, we find that the magnitude of noise is large (10%–30% of forecast MSE) and comparable to bias. We illustrate how our estimates can be applied to calibrate models with incomplete information and bound the effects of measurement error.

Suggested Citation

  • Artūras Juodis & Simas Kučinskas, 2023. "Quantifying noise in survey expectations," Quantitative Economics, Econometric Society, vol. 14(2), pages 609-650, May.
    • Handle: RePEc:wly:quante:v:14:y:2023:i:2:p:609-650
    DOI: 10.3982/QE1633
    as

    Download full text from publisher

    File URL: https://doi.org/10.3982/QE1633
    Download Restriction: no
    File URL: https://libkey.io/10.3982/QE1633?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ottaviani, Marco & Sorensen, Peter Norman, 2006. "The strategy of professional forecasting," Journal of Financial Economics, Elsevier, vol. 81(2), pages 441-466, August.
    2. Jean‐Philippe Bouchaud & Philipp Krüger & Augustin Landier & David Thesmar, 2019. "Sticky Expectations and the Profitability Anomaly," Journal of Finance, American Finance Association, vol. 74(2), pages 639-674, April.
    3. Okui, Ryo & Yanagi, Takahide, 2019. "Panel data analysis with heterogeneous dynamics," Journal of Econometrics, Elsevier, vol. 212(2), pages 451-475.
    4. Pedro Bordalo & Nicola Gennaioli & Yueran Ma & Andrei Shleifer, 2020. "Overreaction in Macroeconomic Expectations," American Economic Review, American Economic Association, vol. 110(9), pages 2748-2782, September.
    5. Olivier Coibion & Yuriy Gorodnichenko, 2012. "What Can Survey Forecasts Tell Us about Information Rigidities?," Journal of Political Economy, University of Chicago Press, vol. 120(1), pages 116-159.
    6. Terrance Odean., 1996. "Volume, Volatility, Price and Profit When All Trader Are Above Average," Research Program in Finance Working Papers RPF-266, University of California at Berkeley.
    7. Charles F. Manski, 2018. "Survey Measurement of Probabilistic Macroeconomic Expectations: Progress and Promise," NBER Macroeconomics Annual, University of Chicago Press, vol. 32(1), pages 411-471.
    8. Bartosz Mackowiak & Mirko Wiederholt, 2009. "Optimal Sticky Prices under Rational Inattention," American Economic Review, American Economic Association, vol. 99(3), pages 769-803, June.
    9. Dovern, Jonas, 2015. "A multivariate analysis of forecast disagreement: Confronting models of disagreement with survey data," European Economic Review, Elsevier, vol. 80(C), pages 16-35.
    10. Chen, Mingli & Fernández-Val, Iván & Weidner, Martin, 2021. "Nonlinear factor models for network and panel data," Journal of Econometrics, Elsevier, vol. 220(2), pages 296-324.
    11. Figlewski, Stephen, 1983. "Optimal Price Forecasting Using Survey Data," The Review of Economics and Statistics, MIT Press, vol. 65(1), pages 13-21, February.
    12. Forni, Mario & Hallin, Marc & Lippi, Marco & Zaffaroni, Paolo, 2015. "Dynamic factor models with infinite-dimensional factor spaces: One-sided representations," Journal of Econometrics, Elsevier, vol. 185(2), pages 359-371.
    13. Artūras Juodis, 2018. "Pseudo Panel Data Models With Cohort Interactive Effects," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 36(1), pages 47-61, January.
    14. Seung C. Ahn & Alex R. Horenstein, 2013. "Eigenvalue Ratio Test for the Number of Factors," Econometrica, Econometric Society, vol. 81(3), pages 1203-1227, May.
    15. Alexei Onatski, 2009. "Testing Hypotheses About the Number of Factors in Large Factor Models," Econometrica, Econometric Society, vol. 77(5), pages 1447-1479, September.
    16. Davies, Anthony & Lahiri, Kajal, 1995. "A new framework for analyzing survey forecasts using three-dimensional panel data," Journal of Econometrics, Elsevier, vol. 68(1), pages 205-227, July.
    17. Andreas Fuster & David Laibson & Brock Mendel, 2010. "Natural Expectations and Macroeconomic Fluctuations," Journal of Economic Perspectives, American Economic Association, vol. 24(4), pages 67-84, Fall.
    18. Tom Stark, 1997. "Macroeconomic forecasts and microeconomic forecasters in the Survey of Professional Forecasters," Working Papers 97-10, Federal Reserve Bank of Philadelphia.
    19. Giustinelli, Pamela & Manski, Charles F. & Molinari, Francesca, 2022. "Tail and center rounding of probabilistic expectations in the Health and Retirement Study," Journal of Econometrics, Elsevier, vol. 231(1), pages 265-281.
    20. Jin, Sainan & Miao, Ke & Su, Liangjun, 2021. "On factor models with random missing: EM estimation, inference, and cross validation," Journal of Econometrics, Elsevier, vol. 222(1), pages 745-777.
    21. Shiller, Robert J, 1981. "Do Stock Prices Move Too Much to be Justified by Subsequent Changes in Dividends?," American Economic Review, American Economic Association, vol. 71(3), pages 421-436, June.
    22. Olivier Coibion & Yuriy Gorodnichenko, 2015. "Information Rigidity and the Expectations Formation Process: A Simple Framework and New Facts," American Economic Review, American Economic Association, vol. 105(8), pages 2644-2678, August.
    23. Bachmann, Rüdiger & Elstner, Steffen, 2015. "Firm optimism and pessimism," European Economic Review, Elsevier, vol. 79(C), pages 297-325.
    24. Pedro Bordalo & Nicola Gennaioli & Andrei Shleifer, 2018. "Diagnostic Expectations and Credit Cycles," Journal of Finance, American Finance Association, vol. 73(1), pages 199-227, February.
    25. Ryan Chahrour & Kyle Jurado, 2018. "News or Noise? The Missing Link," American Economic Review, American Economic Association, vol. 108(7), pages 1702-1736, July.
    26. Forni, Mario & Hallin, Marc & Lippi, Marco & Zaffaroni, Paolo, 2017. "Dynamic factor models with infinite-dimensional factor space: Asymptotic analysis," Journal of Econometrics, Elsevier, vol. 199(1), pages 74-92.
    27. Zhen Huo & Marcelo Pedroni, 2020. "A Single-Judge Solution to Beauty Contests," American Economic Review, American Economic Association, vol. 110(2), pages 526-568, February.
    28. Ang, Andrew & Bekaert, Geert & Wei, Min, 2007. "Do macro variables, asset markets, or surveys forecast inflation better?," Journal of Monetary Economics, Elsevier, vol. 54(4), pages 1163-1212, May.
    29. Chudik, Alexander & Pesaran, M. Hashem, 2015. "Common correlated effects estimation of heterogeneous dynamic panel data models with weakly exogenous regressors," Journal of Econometrics, Elsevier, vol. 188(2), pages 393-420.
    30. Jushan Bai, 2003. "Inferential Theory for Factor Models of Large Dimensions," Econometrica, Econometric Society, vol. 71(1), pages 135-171, January.
    31. Leonardo Melosi, 2014. "Estimating Models with Dispersed Information," American Economic Journal: Macroeconomics, American Economic Association, vol. 6(1), pages 1-31, January.
    32. M. Hashem Pesaran, 2006. "Estimation and Inference in Large Heterogeneous Panels with a Multifactor Error Structure," Econometrica, Econometric Society, vol. 74(4), pages 967-1012, July.
    33. Lucas, Robert E, Jr, 1973. "Some International Evidence on Output-Inflation Tradeoffs," American Economic Review, American Economic Association, vol. 63(3), pages 326-334, June.
    34. Drerup, Tilman & Enke, Benjamin & von Gaudecker, Hans-Martin, 2017. "The precision of subjective data and the explanatory power of economic models," Journal of Econometrics, Elsevier, vol. 200(2), pages 378-389.
    35. Stock, James H & Watson, Mark W, 2002. "Macroeconomic Forecasting Using Diffusion Indexes," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 147-162, April.
    36. Sims, Christopher A., 2003. "Implications of rational inattention," Journal of Monetary Economics, Elsevier, vol. 50(3), pages 665-690, April.
    37. Kyle, Albert S, 1985. "Continuous Auctions and Insider Trading," Econometrica, Econometric Society, vol. 53(6), pages 1315-1335, November.
    38. Jushan Bai, 2009. "Panel Data Models With Interactive Fixed Effects," Econometrica, Econometric Society, vol. 77(4), pages 1229-1279, July.
    39. Terrance Odean, 1998. "Volume, Volatility, Price, and Profit When All Traders Are Above Average," Journal of Finance, American Finance Association, vol. 53(6), pages 1887-1934, December.
    40. Stock, J.H. & Watson, M.W., 2016. "Dynamic Factor Models, Factor-Augmented Vector Autoregressions, and Structural Vector Autoregressions in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 415-525, Elsevier.
    41. Guido Lorenzoni, 2009. "A Theory of Demand Shocks," American Economic Review, American Economic Association, vol. 99(5), pages 2050-2084, December.
    42. Artūras Juodis & Simon Reese, 2022. "The Incidental Parameters Problem in Testing for Remaining Cross-Section Correlation," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 40(3), pages 1191-1203, June.
    43. Michael Bailey & Ruiqing Cao & Theresa Kuchler & Johannes Stroebel, 2018. "The Economic Effects of Social Networks: Evidence from the Housing Market," Journal of Political Economy, University of Chicago Press, vol. 126(6), pages 2224-2276.
    44. Mr. Christopher W. Crowe, 2010. "Consensus Forecasts and Inefficient Information Aggregation," IMF Working Papers 2010/178, International Monetary Fund.
    45. Angeletos, G.-M. & Lian, C., 2016. "Incomplete Information in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 1065-1240, Elsevier.
    46. Juodis, Artūras & Karabiyik, Hande & Westerlund, Joakim, 2021. "On the robustness of the pooled CCE estimator," Journal of Econometrics, Elsevier, vol. 220(2), pages 325-348.
    47. Manski, Charles F. & Molinari, Francesca, 2010. "Rounding Probabilistic Expectations in Surveys," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(2), pages 219-231.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Born, Benjamin & Enders, Zeno & Menkhoff, Manuel & Müller, Gernot & Niemann, Knut, 2022. "Firm Expectations and News: Micro v Macro," CEPR Discussion Papers 17768, C.E.P.R. Discussion Papers.
    2. Bauer, Michael D. & Pflueger, Carolin E. & Sunderam, Adi, 2022. "Perceptions about monetary policy," IMFS Working Paper Series 176, Goethe University Frankfurt, Institute for Monetary and Financial Stability (IMFS).
    3. Born, Benjamin & Enders, Zeno & Müller, Gernot J., 2023. "On FIRE, news, and expectations," Working Papers 42, German Research Foundation's Priority Programme 1859 "Experience and Expectation. Historical Foundations of Economic Behaviour", Humboldt University Berlin.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jean-Paul L'Huillier & Sanjay R. Singh & Donghoon Yoo, 2021. "Incorporating Diagnostic Expectations into the New Keynesian Framework," Working Papers 339, University of California, Davis, Department of Economics.
    2. Angeletos, G.-M. & Lian, C., 2016. "Incomplete Information in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 1065-1240, Elsevier.
    3. Born, Benjamin & Enders, Zeno & Müller, Gernot J., 2023. "On FIRE, news, and expectations," Working Papers 42, German Research Foundation's Priority Programme 1859 "Experience and Expectation. Historical Foundations of Economic Behaviour", Humboldt University Berlin.
    4. Florian Peters & Simas Kucinskas, 2018. "Measuring Biases in Expectation Formation," Tinbergen Institute Discussion Papers 18-058/IV, Tinbergen Institute.
    5. Goldstein, Nathan & Zilberfarb, Ben-Zion, 2021. "Do forecasters really care about consensus?," Economic Modelling, Elsevier, vol. 100(C).
    6. Hagenhoff, Tim & Lustenhouwer, Joep, 2023. "The role of stickiness, extrapolation and past consensus forecasts in macroeconomic expectations," Journal of Economic Dynamics and Control, Elsevier, vol. 149(C).
    7. Catherine Doz & Peter Fuleky, 2019. "Dynamic Factor Models," Working Papers 2019-4, University of Hawaii Economic Research Organization, University of Hawaii at Manoa.
    8. Francesco Bianchi & Sydney C. Ludvigson & Sai Ma, 2022. "Belief Distortions and Macroeconomic Fluctuations," American Economic Review, American Economic Association, vol. 112(7), pages 2269-2315, July.
    9. Yinchu Zhu, 2019. "How well can we learn large factor models without assuming strong factors?," Papers 1910.10382, arXiv.org, revised Nov 2019.
    10. Pedro Bordalo & Nicola Gennaioli & Rafael La Porta & Andrei Shleifer, 2020. "Belief Overreaction and Stock Market Puzzles," NBER Working Papers 27283, National Bureau of Economic Research, Inc.
    11. Artūras Juodis, 2022. "A regularization approach to common correlated effects estimation," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(4), pages 788-810, June.
    12. Catherine Doz & Peter Fuleky, 2019. "Dynamic Factor Models," PSE Working Papers halshs-02262202, HAL.
    13. Catherine Doz & Peter Fuleky, 2019. "Dynamic Factor Models," Working Papers halshs-02262202, HAL.
    14. Alexandros Botsis & Christoph Görtz & Plutarchos Sakellaris, 2020. "Quantifying Qualitative Survey Data: New Insights on the (Ir)Rationality of Firms' Forecasts," CESifo Working Paper Series 8148, CESifo.
    15. Francesco D'Acunto & Ulrike M. Malmendier & Juan Ospina & Michael Weber & Michael Weber, 2019. "Exposure to Daily Price Changes and Inflation Expectations," CESifo Working Paper Series 7798, CESifo.
    16. Hagenhoff, Tim & Lustenhouwer, Joep, 2020. "The role of stickiness, extrapolation and past consensus forecasts in macroeconomic expectations," BERG Working Paper Series 163, Bamberg University, Bamberg Economic Research Group.
    17. Chen, Cheng & Senga, Tatsuro & Sun, Chang & Zhang, Hongyong, 2023. "Uncertainty, imperfect information, and expectation formation over the firm’s life cycle," Journal of Monetary Economics, Elsevier, vol. 140(C), pages 60-77.
    18. Bartosz Maćkowiak & Filip Matějka & Mirko Wiederholt, 2023. "Rational Inattention: A Review," Journal of Economic Literature, American Economic Association, vol. 61(1), pages 226-273, March.
    19. An, Zidong & Zheng, Xinye, 2023. "Diligent forecasters can make accurate predictions despite disagreeing with the consensus," Economic Modelling, Elsevier, vol. 125(C).
    20. George‐Marios Angeletos & Fabrice Collard & Harris Dellas, 2018. "Quantifying Confidence," Econometrica, Econometric Society, vol. 86(5), pages 1689-1726, September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:quante:v:14:y:2023:i:2:p:609-650. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/essssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.