IDEAS home Printed from https://ideas.repec.org/p/bri/uobdis/15-661.html

Counting Biased Forecasters: An Application of Multiple Testing Techniques

Author

Listed:
  • Fabiana Gomez
  • David Pacini

Abstract

We investigate the problem of counting biased forecasters among a group of unbiased and biased forecasters of macroeconomic variables. The innovation is to implement a procedure controlling for the expected proportion of unbiased forecasters that could be erroneously classified as biased (i.e., the false discovery rate). Monte Carlo exercises illustrate the relevance of controlling the false discovery rate in this context. Using data from the Survey of Professional Forecasters, we find that up to 7 out of 10 forecasters classified as biased by a procedure not controlling the false discovery rate may actually be unbiased.

Suggested Citation

  • Fabiana Gomez & David Pacini, 2015. "Counting Biased Forecasters: An Application of Multiple Testing Techniques," Bristol Economics Discussion Papers 15/661, School of Economics, University of Bristol, UK.
    • Handle: RePEc:bri:uobdis:15/661
    as

    Download full text from publisher

    File URL: http://www.bristol.ac.uk/efm/media/workingpapers/working_papers/pdffiles/dp15661.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jacob A. Mincer & Victor Zarnowitz, 1969. "The Evaluation of Economic Forecasts," NBER Chapters, in: Economic Forecasts and Expectations: Analysis of Forecasting Behavior and Performance, pages 3-46, National Bureau of Economic Research, Inc.
    2. Brown, Bryan W & Maital, Shlomo, 1981. "What Do Economists Know? An Empirical Study of Experts' Expectations," Econometrica, Econometric Society, vol. 49(2), pages 491-504, March.
    3. Batchelor, Roy & Dua, Pami, 1991. "Blue Chip Rationality Tests," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 23(4), pages 692-705, November.
    4. Bonham, Carl S & Cohen, Richard H, 2001. "To Aggregate, Pool, or Neither: Testing the Rational-Expectations Hypothesis Using Survey Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(3), pages 278-291, July.
    5. Batchelor, R A, 1990. "All Forecasters Are Equal," Journal of Business & Economic Statistics, American Statistical Association, vol. 8(1), pages 143-144, January.
    6. Scott Schuh, 2001. "An evaluation of recent macroeconomic forecast errors," New England Economic Review, Federal Reserve Bank of Boston, pages 35-56.
    7. James H. Stock & Mark W. Watson, 2007. "Erratum to "Why Has U.S. Inflation Become Harder to Forecast?"," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 39(7), pages 1849-1849, October.
    8. Davies, Anthony & Lahiri, Kajal, 1995. "A new framework for analyzing survey forecasts using three-dimensional panel data," Journal of Econometrics, Elsevier, vol. 68(1), pages 205-227, July.
    9. Ivana Komunjer & Michael T. Owyang, 2012. "Multivariate Forecast Evaluation and Rationality Testing," The Review of Economics and Statistics, MIT Press, vol. 94(4), pages 1066-1080, November.
    10. Keane, Michael P & Runkle, David E, 1990. "Testing the Rationality of Price Forecasts: New Evidence from Panel Data," American Economic Review, American Economic Association, vol. 80(4), pages 714-735, September.
    11. Carlos Capistr¡N & Allan Timmermann, 2009. "Disagreement and Biases in Inflation Expectations," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 41(2-3), pages 365-396, March.
    12. Zarnowitz, Victor, 1985. "Rational Expectations and Macroeconomic Forecasts," Journal of Business & Economic Statistics, American Statistical Association, vol. 3(4), pages 293-311, October.
    13. Antonello D’Agostino & Kieran Mcquinn & Karl Whelan, 2012. "Are Some Forecasters Really Better Than Others?," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 44(4), pages 715-732, June.
    14. Croushore Dean, 2010. "An Evaluation of Inflation Forecasts from Surveys Using Real-Time Data," The B.E. Journal of Macroeconomics, De Gruyter, vol. 10(1), pages 1-32, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michael P. Clements, 2022. "Forecaster Efficiency, Accuracy, and Disagreement: Evidence Using Individual‐Level Survey Data," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 54(2-3), pages 537-568, March.
    2. Fildes, Robert & Stekler, Herman, 2002. "The state of macroeconomic forecasting," Journal of Macroeconomics, Elsevier, vol. 24(4), pages 435-468, December.
    3. Croushore Dean, 2010. "An Evaluation of Inflation Forecasts from Surveys Using Real-Time Data," The B.E. Journal of Macroeconomics, De Gruyter, vol. 10(1), pages 1-32, May.
    4. Graham Elliott & Ivana Komunjer & Allan Timmermann, 2008. "Biases in Macroeconomic Forecasts: Irrationality or Asymmetric Loss?," Journal of the European Economic Association, MIT Press, vol. 6(1), pages 122-157, March.
    5. repec:lan:wpaper:470 is not listed on IDEAS
    6. Gregor W. Smith & James Yetman, 2013. "Estimating Dynamic Euler Equations With Multivariate Professional Forecasts," Economic Inquiry, Western Economic Association International, vol. 51(1), pages 445-458, January.
    7. Clements, Michael P., 2021. "Do survey joiners and leavers differ from regular participants? The US SPF GDP growth and inflation forecasts," International Journal of Forecasting, Elsevier, vol. 37(2), pages 634-646.
    8. repec:spo:wpecon:info:hdl:2441/f4rshpf3v1umfa09lat09b1bg is not listed on IDEAS
    9. repec:lan:wpaper:425 is not listed on IDEAS
    10. repec:lan:wpaper:539557 is not listed on IDEAS
    11. repec:lan:wpaper:413 is not listed on IDEAS
    12. Carl Bonham & Richard Cohen & Shigeyuki Abe, 2006. "The Rationality and Heterogeneity of Survey Forecasts of the Yen-Dollar Exchange Rate: A Reexamination," Working Papers 200611, University of Hawaii at Manoa, Department of Economics.
    13. Silva Lopes, Artur, 1994. "A "hipótese das expectativas racionais": teoria e realidade (uma visita guiada à literatura até 1992) [The "rational expectations hypothesis": theory and reality (a guided tour to the literature published until 1992)]," MPRA Paper 9699, University Library of Munich, Germany, revised 23 Jul 2008.
    14. Paul Hubert, 2010. "Monetary policy, imperfect information and the expectations channel [Politique monétaire,information imparfaite et canal des anticipations]," Sciences Po Economics Publications (main) tel-04095385, HAL.
    15. repec:spo:wpmain:info:hdl:2441/f4rshpf3v1umfa09lat09b1bg is not listed on IDEAS
    16. Julieta Caunedo & Riccardo Dicecio & Ivana Komunjer & Michael T. Owyang, 2020. "Asymmetry, Complementarities, and State Dependence in Federal Reserve Forecasts," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 52(1), pages 205-228, February.
    17. Smith, Gregor W. & Yetman, James, 2007. "The Curse of Irving Fisher (Professional Forecasters’ Version)," Queen's Economics Department Working Papers 273620, Queen's University - Department of Economics.
    18. Isiklar, Gultekin, 2005. "On aggregation bias in fixed-event forecast efficiency tests," Economics Letters, Elsevier, vol. 89(3), pages 312-316, December.
    19. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    20. Nathan Goldstein & Ben‐Zion Zilberfarb, 2023. "The closer we get, the better we are?," Economic Inquiry, Western Economic Association International, vol. 61(2), pages 364-376, April.
    21. Antonello D’Agostino & Kieran Mcquinn & Karl Whelan, 2012. "Are Some Forecasters Really Better Than Others?," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 44(4), pages 715-732, June.
    22. Davies, Anthony & Lahiri, Kajal, 1995. "A new framework for analyzing survey forecasts using three-dimensional panel data," Journal of Econometrics, Elsevier, vol. 68(1), pages 205-227, July.
    23. Bonham, Carl S & Cohen, Richard H, 2001. "To Aggregate, Pool, or Neither: Testing the Rational-Expectations Hypothesis Using Survey Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(3), pages 278-291, July.
    24. Jordi Pons-Novell, 2003. "Strategic bias, herding behaviour and economic forecasts," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 22(1), pages 67-77.
    25. Clements, Michael P., "undated". "Internal consistency of survey respondentsíforecasts: Evidence based on the Survey of Professional Forecasters," Economic Research Papers 269742, University of Warwick - Department of Economics.
    26. Ager, P. & Kappler, M. & Osterloh, S., 2009. "The accuracy and efficiency of the Consensus Forecasts: A further application and extension of the pooled approach," International Journal of Forecasting, Elsevier, vol. 25(1), pages 167-181.

    More about this item

    Keywords

    ;
    ;
    ;

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models
    • E17 - Macroeconomics and Monetary Economics - - General Aggregative Models - - - Forecasting and Simulation: Models and Applications

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bri:uobdis:15/661. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: School of Economics Research Support Team (email available below). General contact details of provider: https://edirc.repec.org/data/sebriuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.