IDEAS home Printed from https://ideas.repec.org/a/spr/jbuscr/v19y2023i2d10.1007_s41549-023-00082-4.html
   My bibliography  Save this article

ROC and PRC Approaches to Evaluate Recession Forecasts

Author

Listed:
  • Kajal Lahiri

    (University at Abany: SUNY)

  • Cheng Yang

    (Liaoning University)

Abstract

We have studied the relationship between Receiver Operating Characteristic (ROC) curve and Precision-Recall Curve (PRC) both analytically and using a real-life empirical example of yield spread as a predictor of recessions. We show that false alarm rate in ROC and inverted precision in PRC are analogous concepts, and their difference is determined by the interaction of sample imbalance and forecast bias. We found that in cases of severe class imbalance, the forecasts need to be adequately biased to mitigate the effect of imbalancedness. The mix of values of precision and recall over six sub-samples show that the predictive power of the spread has not deteriorated in recent decades, provided the optimum values of threshold are used. Using PRC, we quantify the extent to which ROC could be exaggerating the true predictive value of the yield curve in predicting recessions.

Suggested Citation

  • Kajal Lahiri & Cheng Yang, 2023. "ROC and PRC Approaches to Evaluate Recession Forecasts," Journal of Business Cycle Research, Springer;Centre for International Research on Economic Tendency Surveys (CIRET), vol. 19(2), pages 119-148, September.
    • Handle: RePEc:spr:jbuscr:v:19:y:2023:i:2:d:10.1007_s41549-023-00082-4
    DOI: 10.1007/s41549-023-00082-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s41549-023-00082-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s41549-023-00082-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Takaya Saito & Marc Rehmsmeier, 2015. "The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets," PLOS ONE, Public Library of Science, vol. 10(3), pages 1-21, March.
    2. Arturo Estrella & Frederic S. Mishkin, 1996. "The yield curve as a predictor of U.S. recessions," Current Issues in Economics and Finance, Federal Reserve Bank of New York, vol. 2(Jun).
    3. Kajal Lahiri & Cheng Yang, 2022. "ROC approach to forecasting recessions using daily yield spreads," Business Economics, Palgrave Macmillan;National Association for Business Economics, vol. 57(4), pages 191-203, October.
    4. Jonathan Cook & Vikram Ramadas, 2020. "When to consult precision-recall curves," Stata Journal, StataCorp LP, vol. 20(1), pages 131-148, March.
    5. Travis J. Berge & Òscar Jordà, 2011. "Evaluating the Classification of Economic Activity into Recessions and Expansions," American Economic Journal: Macroeconomics, American Economic Association, vol. 3(2), pages 246-277, April.
    6. Raffaella Giacomini & Barbara Rossi, 2006. "How Stable is the Forecasting Performance of the Yield Curve for Output Growth?," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 68(s1), pages 783-795, December.
    7. James H. Stock & Mark W. Watson, 1993. "Business Cycles, Indicators, and Forecasting," NBER Books, National Bureau of Economic Research, Inc, number stoc93-1, March.
    8. Lahiri, Kajal & Monokroussos, George & Zhao, Yongchen, 2013. "The yield spread puzzle and the information content of SPF forecasts," Economics Letters, Elsevier, vol. 118(1), pages 219-221.
    9. Marcelle Chauvet & Simon Potter, 2005. "Forecasting recessions using the yield curve," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 24(2), pages 77-103.
    10. Jaehyuk Choi & Desheng Ge & Kyu Ho Kang & Sungbin Sohn, 2023. "Yield spread selection in predicting recession probabilities," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(7), pages 1772-1785, November.
    11. Robert P. Lieli & Yu-Chin Hsu, 2019. "Using the area under an estimated ROC curve to test the adequacy of binary predictors," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 31(1), pages 100-130, January.
    12. Jardet, Caroline, 2004. "Why did the term structure of interest rates lose its predictive power?," Economic Modelling, Elsevier, vol. 21(3), pages 509-524, May.
    13. Lahiri, Kajal & Yang, Liu, 2013. "Forecasting Binary Outcomes," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1025-1106, Elsevier.
    14. Lahiri Kajal & Yang Liu, 2021. "Construction of leading economic index for recession prediction using vine copulas," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 25(4), pages 193-212, September.
    15. Vrontos, Spyridon D. & Galakis, John & Vrontos, Ioannis D., 2021. "Modeling and predicting U.S. recessions using machine learning techniques," International Journal of Forecasting, Elsevier, vol. 37(2), pages 647-671.
    16. Estrella, Arturo & Hardouvelis, Gikas A, 1991. "The Term Structure as a Predictor of Real Economic Activity," Journal of Finance, American Finance Association, vol. 46(2), pages 555-576, June.
    17. Stock, James H. & Watson, Mark W. (ed.), 1993. "Business Cycles, Indicators, and Forecasting," National Bureau of Economic Research Books, University of Chicago Press, edition 1, number 9780226774886, December.
    18. Elliott, Graham & Lieli, Robert P., 2013. "Predicting binary outcomes," Journal of Econometrics, Elsevier, vol. 174(1), pages 15-26.
    19. Jens Keilwagen & Ivo Grosse & Jan Grau, 2014. "Area under Precision-Recall Curves for Weighted and Unweighted Data," PLOS ONE, Public Library of Science, vol. 9(3), pages 1-13, March.
    20. Michael Dotsey, 1998. "The predictive content of the interest rate term spread for future economic growth," Economic Quarterly, Federal Reserve Bank of Richmond, issue Sum, pages 31-51.
    21. Levanon, Gad & Manini, Jean-Claude & Ozyildirim, Ataman & Schaitkin, Brian & Tanchua, Jennelyn, 2015. "Using financial indicators to predict turning points in the business cycle: The case of the leading economic index for the United States," International Journal of Forecasting, Elsevier, vol. 31(2), pages 426-445.
    22. Rudebusch, Glenn D. & Williams, John C., 2009. "Forecasting Recessions: The Puzzle of the Enduring Power of the Yield Curve," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(4), pages 492-503.
    23. Michael D. Bauer & Thomas M. Mertens, 2018. "Information in the Yield Curve about Future Recessions," FRBSF Economic Letter, Federal Reserve Bank of San Francisco.
    24. Daniel H. Cooper & Jeffrey C. Fuhrer & Giovanni P. Olivei, 2020. "Predicting Recessions Using the Yield Curve: The Role of the Stance of Monetary Policy," Current Policy Perspectives 87522, Federal Reserve Bank of Boston.
    25. Kajal Lahiri & Cheng Yang, 2023. "A tale of two recession-derivative indicators," Empirical Economics, Springer, vol. 65(2), pages 925-947, August.
    26. James H. Stock & Mark W. Watson, 1993. "Introduction to "Business Cycles, Indicators and Forecasting"," NBER Chapters, in: Business Cycles, Indicators, and Forecasting, pages 1-10, National Bureau of Economic Research, Inc.
    27. Rummens, Anneleen & Hardyns, Wim, 2021. "The effect of spatiotemporal resolution on predictive policing model performance," International Journal of Forecasting, Elsevier, vol. 37(1), pages 125-133.
    28. Peter Johansson & Andrew C. Meldrum, 2018. "Predicting Recession Probabilities Using the Slope of the Yield Curve," FEDS Notes 2018-03-01-3, Board of Governors of the Federal Reserve System (U.S.).
    29. Pigini, Claudia, 2021. "Penalized maximum likelihood estimation of logit-based early warning systems," International Journal of Forecasting, Elsevier, vol. 37(3), pages 1156-1172.
    30. Andrea Ajello & Luca Benzoni & Makena Schwinn & Yannick Timmer & Francisco Vazquez-Grande, 2022. "Monetary Policy, Inflation Outlook, and Recession Probabilities," FEDS Notes 2022-07-12, Board of Governors of the Federal Reserve System (U.S.).
    31. Lin, Fan & Zhang, Yao & Wang, Jianxue, 2023. "Recent advances in intra-hour solar forecasting: A review of ground-based sky image methods," International Journal of Forecasting, Elsevier, vol. 39(1), pages 244-265.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kajal Lahiri & Cheng Yang, 2022. "ROC approach to forecasting recessions using daily yield spreads," Business Economics, Palgrave Macmillan;National Association for Business Economics, vol. 57(4), pages 191-203, October.
    2. B. De Backer & M. Deroose & Ch. Van Nieuwenhuyze, 2019. "Is a recession imminent? The signal of the yield curve," Economic Review, National Bank of Belgium, issue i, pages 69-93, June.
    3. Liu, Weiling & Moench, Emanuel, 2016. "What predicts US recessions?," International Journal of Forecasting, Elsevier, vol. 32(4), pages 1138-1150.
    4. Joseph G. Haubrich, 2021. "Does the Yield Curve Predict Output?," Annual Review of Financial Economics, Annual Reviews, vol. 13(1), pages 341-362, November.
    5. Kajal Lahiri & Cheng Yang, 2023. "A tale of two recession-derivative indicators," Empirical Economics, Springer, vol. 65(2), pages 925-947, August.
    6. Knut Lehre Seip & Dan Zhang, 2021. "The Yield Curve as a Leading Indicator: Accuracy and Timing of a Parsimonious Forecasting Model," Forecasting, MDPI, vol. 3(2), pages 1-16, May.
    7. Massimo Ferrari Minesso & Laura Lebastard & Helena Mezo, 2023. "Text-Based Recession Probabilities," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 71(2), pages 415-438, June.
    8. Heikki Kauppi, 2019. "Recession Prediction with OptimalUse of Leading Indicators," Discussion Papers 125, Aboa Centre for Economics.
    9. Lahiri, Kajal & Yang, Liu, 2013. "Forecasting Binary Outcomes," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1025-1106, Elsevier.
    10. Hasse, Jean-Baptiste & Lajaunie, Quentin, 2022. "Does the yield curve signal recessions? New evidence from an international panel data analysis," The Quarterly Review of Economics and Finance, Elsevier, vol. 84(C), pages 9-22.
    11. Shuping Shi & Peter C. B. Phillips & Stan Hurn, 2018. "Change Detection and the Causal Impact of the Yield Curve," Journal of Time Series Analysis, Wiley Blackwell, vol. 39(6), pages 966-987, November.
    12. Jonathan H. Wright, 2006. "The yield curve and predicting recessions," Finance and Economics Discussion Series 2006-07, Board of Governors of the Federal Reserve System (U.S.).
    13. Aguiar-Conraria, Luís & Martins, Manuel M.F. & Soares, Maria Joana, 2012. "The yield curve and the macro-economy across time and frequencies," Journal of Economic Dynamics and Control, Elsevier, vol. 36(12), pages 1950-1970.
    14. Hwang, Youngjin, 2019. "Forecasting recessions with time-varying models," Journal of Macroeconomics, Elsevier, vol. 62(C).
    15. Herman O. Stekler & Tianyu Ye, 2017. "Evaluating a leading indicator: an application—the term spread," Empirical Economics, Springer, vol. 53(1), pages 183-194, August.
    16. Vasilios Plakandaras & Juncal Cunado & Rangan Gupta & Mark E. Wohar, 2016. "Do Leading Indicators Forecast U.S. Recessions? A Nonlinear Re-Evaluation Using Historical Data," Working Papers 201685, University of Pretoria, Department of Economics.
    17. Jaehyuk Choi & Desheng Ge & Kyu Ho Kang & Sungbin Sohn, 2021. "Yield Spread Selection in Predicting Recession Probabilities: A Machine Learning Approach," Papers 2101.09394, arXiv.org, revised Jan 2022.
    18. Sergey V. Smirnov & Nikolay V. Kondrashov & Anna V. Petronevich, 2017. "Dating Cyclical Turning Points for Russia: Formal Methods and Informal Choices," Journal of Business Cycle Research, Springer;Centre for International Research on Economic Tendency Surveys (CIRET), vol. 13(1), pages 53-73, May.
    19. Aastveit, Knut Are & Anundsen, André K. & Herstad, Eyo I., 2019. "Residential investment and recession predictability," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1790-1799.
    20. J.D. Hollingworth, 1997. "Leading Indicators of Australian Recessions: Part 2," Economics Discussion / Working Papers 97-17, The University of Western Australia, Department of Economics.

    More about this item

    Keywords

    Business cycle; NBER; Yield spread; ROC; PRC; Recession;
    All these keywords.

    JEL classification:

    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • E17 - Macroeconomics and Monetary Economics - - General Aggregative Models - - - Forecasting and Simulation: Models and Applications
    • E27 - Macroeconomics and Monetary Economics - - Consumption, Saving, Production, Employment, and Investment - - - Forecasting and Simulation: Models and Applications
    • E32 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Business Fluctuations; Cycles

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:jbuscr:v:19:y:2023:i:2:d:10.1007_s41549-023-00082-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.