IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v29y2013i1p175-190.html
   My bibliography  Save this article

Evaluating probability forecasts for GDP declines using alternative methodologies

Author

Listed:
  • Lahiri, Kajal
  • Wang, J. George

Abstract

Evaluation methodologies for rare events from meteorology, psychology and medical diagnosis are used to examine the value of probabilistic forecasts of real GDP declines during the current quarter (Q0) and each of the next four quarters (Q1–Q4) using data from the Survey of Professional Forecasters. We study the quality of these probability forecasts in terms of their calibration, resolution and odds ratio, as well as the relative operating characteristic (ROC) and alternative variance decompositions. Only the shorter-term forecasts (Q0–Q2) are found to possess significant skill in terms of all measures considered, even though they are characterized by an excess of variability and a lack of calibration.

Suggested Citation

  • Lahiri, Kajal & Wang, J. George, 2013. "Evaluating probability forecasts for GDP declines using alternative methodologies," International Journal of Forecasting, Elsevier, vol. 29(1), pages 175-190.
    • Handle: RePEc:eee:intfor:v:29:y:2013:i:1:p:175-190
    DOI: 10.1016/j.ijforecast.2012.07.004
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169207012001094
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ijforecast.2012.07.004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Peter G. Hall & Rob J. Hyndman & Yanan Fan, 2003. "Non Parametric Confidence Intervals for Receiver Operating Characteristic Curves," Monash Econometrics and Business Statistics Working Papers 12/03, Monash University, Department of Econometrics and Business Statistics.
    2. Galbraith, John W. & van Norden, Simon, 2011. "Kernel-based calibration diagnostics for recession and inflation probability forecasts," International Journal of Forecasting, Elsevier, vol. 27(4), pages 1041-1057, October.
    3. Fildes, Robert & Stekler, Herman, 2002. "The state of macroeconomic forecasting," Journal of Macroeconomics, Elsevier, vol. 24(4), pages 435-468, December.
    4. Pesaran, M Hashem & Timmermann, Allan, 1992. "A Simple Nonparametric Test of Predictive Performance," Journal of Business & Economic Statistics, American Statistical Association, vol. 10(4), pages 561-565, October.
    5. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    6. Fildes, Robert & Stekler, Herman, 2002. "Reply to the comments on 'The state of macroeconomic forecasting'," Journal of Macroeconomics, Elsevier, vol. 24(4), pages 503-505, December.
    7. Graham Elliott & Allan Timmermann, 2016. "Economic Forecasting," Economics Books, Princeton University Press, edition 1, number 10740.
    8. Clements, Michael P., 2008. "Consensus and uncertainty: Using forecast probabilities of output declines," International Journal of Forecasting, Elsevier, vol. 24(1), pages 76-86.
    9. Lopez, Jose A, 2001. "Evaluating the Predictive Accuracy of Volatility Models," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 20(2), pages 87-109, March.
    10. Michael P. Clements, 2011. "An Empirical Investigation of the Effects of Rounding on the SPF Probabilities of Decline and Output Growth Histograms," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 43(1), pages 207-220, February.
    11. Isiklar, Gultekin & Lahiri, Kajal, 2007. "How far ahead can we forecast? Evidence from cross-country surveys," International Journal of Forecasting, Elsevier, vol. 23(2), pages 167-187.
    12. Peter Hall, 2004. "Nonparametric confidence intervals for receiver operating characteristic curves," Biometrika, Biometrika Trust, vol. 91(3), pages 743-750, September.
    13. Travis J. Berge & Òscar Jordà, 2011. "Evaluating the Classification of Economic Activity into Recessions and Expansions," American Economic Journal: Macroeconomics, American Economic Association, vol. 3(2), pages 246-277, April.
    14. Pesaran, M. Hashem & Timmermann, Allan, 2009. "Testing Dependence Among Serially Correlated Multicategory Variables," Journal of the American Statistical Association, American Statistical Association, vol. 104(485), pages 325-337.
    15. Granger, Clive W J, 1996. "Can We Improve the Perceived Quality of Economic Forecasts?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 11(5), pages 455-473, Sept.-Oct.
    16. Roopesh Ranjan & Tilmann Gneiting, 2010. "Combining probability forecasts," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 72(1), pages 71-91, January.
    17. Zellner, Arnold & Hong, Chansik & Min, Chung-ki, 1991. "Forecasting turning points in international output growth rates using Bayesian exponentially weighted autoregression, time-varying parameter, and pooling techniques," Journal of Econometrics, Elsevier, vol. 49(1-2), pages 275-304.
    18. Michael P. Clements & David I. Harvey, 2010. "Forecast encompassing tests and probability forecasts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(6), pages 1028-1062.
    19. Andreas Blöchlinger & Markus Leippold, 2011. "A New Goodness-of-Fit Test for Event Forecasting and Its Application to Credit Defaults," Management Science, INFORMS, vol. 57(3), pages 487-505, March.
    20. Graham, John R, 1996. "Is a Group of Economists Better than One? Than None?," The Journal of Business, University of Chicago Press, vol. 69(2), pages 193-232, April.
    21. Rudebusch, Glenn D. & Williams, John C., 2009. "Forecasting Recessions: The Puzzle of the Enduring Power of the Yield Curve," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(4), pages 492-503.
    22. Stein, Roger M., 2005. "The relationship between default prediction and lending profits: Integrating ROC analysis and loan pricing," Journal of Banking & Finance, Elsevier, vol. 29(5), pages 1213-1236, May.
    23. Murphy, Allan H. & Winkler, Robert L., 1992. "Diagnostic verification of probability forecasts," International Journal of Forecasting, Elsevier, vol. 7(4), pages 435-455, March.
    24. Kajal Lahiri & J George Wang, 2006. "Subjective Probability Forecasts for Recessions," Business Economics, Palgrave Macmillan;National Association for Business Economics, vol. 41(2), pages 26-37, April.
    25. Manski, Charles F. & Molinari, Francesca, 2010. "Rounding Probabilistic Expectations in Surveys," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(2), pages 219-231.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lahiri, Kajal & Yang, Liu, 2013. "Forecasting Binary Outcomes," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1025-1106, Elsevier.
    2. Lahiri, Kajal & Peng, Huaming & Zhao, Yongchen, 2015. "Testing the value of probability forecasts for calibrated combining," International Journal of Forecasting, Elsevier, vol. 31(1), pages 113-129.
    3. Sergey V. Smirnov & Daria A. Avdeeva, 2016. "Wishful Bias in Predicting Us Recessions: Indirect Evidence," HSE Working papers WP BRP 135/EC/2016, National Research University Higher School of Economics.
    4. Kajal Lahiri & Liu Yang, 2018. "Confidence Bands for ROC Curves With Serially Dependent Data," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 36(1), pages 115-130, January.
    5. Huiwen Lai & Eric C. Y. Ng, 2020. "On business cycle forecasting," Frontiers of Business Research in China, Springer, vol. 14(1), pages 1-26, December.
    6. G. Kontogeorgos & K. Lambrias, 2022. "Evaluating the Eurosystem/ECB staff macroeconomic projections: The first 20 years," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(2), pages 213-229, March.
    7. Francis X. Diebold & Jose A. Lopez, 1995. "Forecast evaluation and combination," Research Paper 9525, Federal Reserve Bank of New York.
    8. Nathan Goldstein & Ben‐Zion Zilberfarb, 2023. "The closer we get, the better we are?," Economic Inquiry, Western Economic Association International, vol. 61(2), pages 364-376, April.
    9. Turgut Kisinbay & Chikako Baba, 2011. "Predicting Recessions: A New Approach for Identifying Leading Indicators and Forecast Combinations," IMF Working Papers 2011/235, International Monetary Fund.
    10. Marc-Oliver Pohle, 2020. "The Murphy Decomposition and the Calibration-Resolution Principle: A New Perspective on Forecast Evaluation," Papers 2005.01835, arXiv.org.
    11. Michael P. Clements, 2011. "An Empirical Investigation of the Effects of Rounding on the SPF Probabilities of Decline and Output Growth Histograms," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 43(1), pages 207-220, February.
    12. Mitchell, Karlyn & Pearce, Douglas K., 2007. "Professional forecasts of interest rates and exchange rates: Evidence from the Wall Street Journal's panel of economists," Journal of Macroeconomics, Elsevier, vol. 29(4), pages 840-854, December.
    13. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    14. Xie, Zixiong & Hsu, Shih-Hsun, 2016. "Time varying biases and the state of the economy," International Journal of Forecasting, Elsevier, vol. 32(3), pages 716-725.
    15. Henri Nyberg, 2018. "Forecasting US interest rates and business cycle with a nonlinear regime switching VAR model," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 37(1), pages 1-15, January.
    16. Lahiri, Kajal & Yang, Liu, 2016. "Asymptotic variance of Brier (skill) score in the presence of serial correlation," Economics Letters, Elsevier, vol. 141(C), pages 125-129.
    17. Hamid Baghestani, 2011. "A directional analysis of Federal Reserve predictions of growth in unit labor costs and productivity," International Review of Applied Economics, Taylor & Francis Journals, vol. 25(3), pages 303-311.
    18. Kappler, Marcus, 2007. "Projecting the Medium-Term: Outcomes and Errors for GDP Growth," ZEW Discussion Papers 07-068, ZEW - Leibniz Centre for European Economic Research.
    19. Ullrich Heilemann & Herman Stekler, 2010. "Perspectives on Evaluating Macroeconomic Forecasts," Working Papers 2010-002, The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting.
    20. Karsten Müller, 2022. "German forecasters’ narratives: How informative are German business cycle forecast reports?," Empirical Economics, Springer, vol. 62(5), pages 2373-2415, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:29:y:2013:i:1:p:175-190. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.