IDEAS home Printed from https://ideas.repec.org/a/eee/jeborg/v224y2024icp338-354.html
   My bibliography  Save this article

Survey expectations and adjustments for multiple testing

Author

Listed:
  • Clements, Michael P.

Abstract

Testing hypotheses regarding how individual survey respondents form their expectations is susceptible to the multiple testing problem. The probability of falsely rejecting the null hypothesis for one or more respondents will exceed the nominal single-hypothesis significance level. The Bonferroni correction and related approaches control the family-wise error rate, but are conservative and result in low power when the null hypotheses are false.

Suggested Citation

  • Clements, Michael P., 2024. "Survey expectations and adjustments for multiple testing," Journal of Economic Behavior & Organization, Elsevier, vol. 224(C), pages 338-354.
    • Handle: RePEc:eee:jeborg:v:224:y:2024:i:c:p:338-354
    DOI: 10.1016/j.jebo.2024.06.009
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167268124002270
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jebo.2024.06.009?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Joseph P. Romano & Michael Wolf, 2005. "Stepwise Multiple Testing as Formalized Data Snooping," Econometrica, Econometric Society, vol. 73(4), pages 1237-1282, July.
    2. Jacob A. Mincer & Victor Zarnowitz, 1969. "The Evaluation of Economic Forecasts," NBER Chapters, in: Economic Forecasts and Expectations: Analysis of Forecasting Behavior and Performance, pages 3-46, National Bureau of Economic Research, Inc.
    3. Pedro Bordalo & Nicola Gennaioli & Yueran Ma & Andrei Shleifer, 2020. "Overreaction in Macroeconomic Expectations," American Economic Review, American Economic Association, vol. 110(9), pages 2748-2782, September.
    4. Olivier Coibion & Yuriy Gorodnichenko, 2012. "What Can Survey Forecasts Tell Us about Information Rigidities?," Journal of Political Economy, University of Chicago Press, vol. 120(1), pages 116-159.
    5. Tarun Chordia & Amit Goyal & Alessio Saretto & Andrew KarolyiEditor, 2020. "Anomalies and False Rejections," Review of Finance, European Finance Association, vol. 33(5), pages 2134-2179.
    6. Engelberg, Joseph & Manski, Charles F. & Williams, Jared, 2009. "Comparing the Point Predictions and Subjective Probability Distributions of Professional Forecasters," Journal of Business & Economic Statistics, American Statistical Association, vol. 27, pages 30-41.
    7. Croushore, Dean & Stark, Tom, 2001. "A real-time data set for macroeconomists," Journal of Econometrics, Elsevier, vol. 105(1), pages 111-130, November.
    8. N. Gregory Mankiw & Ricardo Reis, 2002. "Sticky Information versus Sticky Prices: A Proposal to Replace the New Keynesian Phillips Curve," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 117(4), pages 1295-1328.
    9. Laurent Barras & Olivier Scaillet & Russ Wermers, 2010. "False Discoveries in Mutual Fund Performance: Measuring Luck in Estimated Alphas," Journal of Finance, American Finance Association, vol. 65(1), pages 179-216, February.
    10. N. Gregory Mankiw & Ricardo Reis & Justin Wolfers, 2004. "Disagreement about Inflation Expectations," NBER Chapters, in: NBER Macroeconomics Annual 2003, Volume 18, pages 209-270, National Bureau of Economic Research, Inc.
    11. Michael P. Clements, 2022. "Individual forecaster perceptions of the persistence of shocks to GDP," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(3), pages 640-656, April.
    12. Michael P. Clements, 2014. "Forecast Uncertainty- Ex Ante and Ex Post : U.S. Inflation and Output Growth," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 32(2), pages 206-216, April.
    13. Olivier Coibion & Yuriy Gorodnichenko, 2015. "Information Rigidity and the Expectations Formation Process: A Simple Framework and New Facts," American Economic Review, American Economic Association, vol. 105(8), pages 2644-2678, August.
    14. Monica Jain, 2019. "Perceived Inflation Persistence," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 37(1), pages 110-120, January.
    15. Mr. John C Bluedorn & Mr. Daniel Leigh, 2018. "Is the Cycle the Trend? Evidence From the Views of International Forecasters," IMF Working Papers 2018/163, International Monetary Fund.
    16. Spencer D. Krane, 2011. "Professional Forecasters' View of Permanent and Transitory Shocks to GDP," American Economic Journal: Macroeconomics, American Economic Association, vol. 3(1), pages 184-211, January.
    17. Dean Croushore, 1993. "Introducing: the survey of professional forecasters," Business Review, Federal Reserve Bank of Philadelphia, issue Nov, pages 3-15.
    18. Clements, Michael P., 2024. "Do professional forecasters believe in the Phillips curve?," International Journal of Forecasting, Elsevier, vol. 40(3), pages 1238-1254.
    19. Jeffrey C. Fuhrer, 2018. "Intrinsic expectations persistence: evidence from professional and household survey expectations," Working Papers 18-9, Federal Reserve Bank of Boston.
    20. Sims, Christopher A., 2003. "Implications of rational inattention," Journal of Monetary Economics, Elsevier, vol. 50(3), pages 665-690, April.
    21. Tarun Chordia & Amit Goyal & Alessio Saretto, 2020. "Anomalies and False Rejections," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2134-2179.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Clements, Michael P., 2024. "Do professional forecasters believe in the Phillips curve?," International Journal of Forecasting, Elsevier, vol. 40(3), pages 1238-1254.
    2. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    3. Clements, Michael P., 2019. "Do forecasters target first or later releases of national accounts data?," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1240-1249.
    4. Michael P. Clements, 2014. "US Inflation Expectations and Heterogeneous Loss Functions, 1968–2010," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 33(1), pages 1-14, January.
    5. Conrad, Christian & Lahiri, Kajal, 2024. "Heterogeneous Expectations among Professional Forecasters," Working Papers 0754, University of Heidelberg, Department of Economics.
    6. Czudaj, Robert L., 2022. "Heterogeneity of beliefs and information rigidity in the crude oil market: Evidence from survey data," European Economic Review, Elsevier, vol. 143(C).
    7. Clements, Michael P., 2021. "Do survey joiners and leavers differ from regular participants? The US SPF GDP growth and inflation forecasts," International Journal of Forecasting, Elsevier, vol. 37(2), pages 634-646.
    8. Andrade, Philippe & Gautier, Erwan & Mengus, Eric, 2023. "What matters in households’ inflation expectations?," Journal of Monetary Economics, Elsevier, vol. 138(C), pages 50-68.
    9. Florian Peters & Simas Kucinskas, 2018. "Measuring Biases in Expectation Formation," Tinbergen Institute Discussion Papers 18-058/IV, Tinbergen Institute.
    10. Gaglianone, Wagner Piazza & Giacomini, Raffaella & Issler, João Victor & Skreta, Vasiliki, 2022. "Incentive-driven inattention," Journal of Econometrics, Elsevier, vol. 231(1), pages 188-212.
    11. Conrad, Christian & Lahiri, Kajal, 2023. "Heterogeneous expectations among professional forecasters," ZEW Discussion Papers 23-062, ZEW - Leibniz Centre for European Economic Research.
    12. Michael Clements, 2016. "Are Macro-Forecasters Essentially The Same? An Analysis of Disagreement, Accuracy and Efficiency," ICMA Centre Discussion Papers in Finance icma-dp2016-08, Henley Business School, University of Reading.
    13. An, Zidong & Zheng, Xinye, 2023. "Diligent forecasters can make accurate predictions despite disagreeing with the consensus," Economic Modelling, Elsevier, vol. 125(C).
    14. Cole, Stephen J. & Milani, Fabio, 2021. "Heterogeneity in individual expectations, sentiment, and constant-gain learning," Journal of Economic Behavior & Organization, Elsevier, vol. 188(C), pages 627-650.
    15. James M. Nason & Gregor W. Smith, 2021. "Measuring the slowly evolving trend in US inflation with professional forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 36(1), pages 1-17, January.
    16. Berge, Travis J., 2018. "Understanding survey-based inflation expectations," International Journal of Forecasting, Elsevier, vol. 34(4), pages 788-801.
    17. Zohar, Osnat, 2024. "Cyclicality of uncertainty and disagreement," Journal of Monetary Economics, Elsevier, vol. 143(C).
    18. Candia, Bernardo & Coibion, Olivier & Gorodnichenko, Yuriy, 2024. "The inflation expectations of U.S. firms: Evidence from a new survey," Journal of Monetary Economics, Elsevier, vol. 145(S).
    19. Hagenhoff, Tim & Lustenhouwer, Joep, 2023. "The role of stickiness, extrapolation and past consensus forecasts in macroeconomic expectations," Journal of Economic Dynamics and Control, Elsevier, vol. 149(C).
    20. Clements, Michael P., 2012. "Do professional forecasters pay attention to data releases?," International Journal of Forecasting, Elsevier, vol. 28(2), pages 297-308.

    More about this item

    Keywords

    Multiple tests; Survey expectations; Family-wise error rates; False discovery rates;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • E37 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Forecasting and Simulation: Models and Applications

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jeborg:v:224:y:2024:i:c:p:338-354. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jebo .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.