IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2003.02803.html
   My bibliography  Save this paper

Equal Predictive Ability Tests Based on Panel Data with Applications to OECD and IMF Forecasts

Author

Listed:
  • Oguzhan Akgun
  • Alain Pirotte
  • Giovanni Urga
  • Zhenlin Yang

Abstract

We propose two types of equal predictive ability (EPA) tests with panels to compare the predictions made by two forecasters. The first type, namely $S$-statistics, focuses on the overall EPA hypothesis which states that the EPA holds on average over all panel units and over time. The second, called $C$-statistics, focuses on the clustered EPA hypothesis where the EPA holds jointly for a fixed number of clusters of panel units. The asymptotic properties of the proposed tests are evaluated under weak and strong cross-sectional dependence. An extensive Monte Carlo simulation shows that the proposed tests have very good finite sample properties even with little information about the cross-sectional dependence in the data. The proposed framework is applied to compare the economic growth forecasts of the OECD and the IMF, and to evaluate the performance of the consumer price inflation forecasts of the IMF.

Suggested Citation

  • Oguzhan Akgun & Alain Pirotte & Giovanni Urga & Zhenlin Yang, 2020. "Equal Predictive Ability Tests Based on Panel Data with Applications to OECD and IMF Forecasts," Papers 2003.02803, arXiv.org, revised Feb 2023.
    • Handle: RePEc:arx:papers:2003.02803
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2003.02803
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Clark, Todd E. & McCracken, Michael W., 2015. "Nested forecast model comparisons: A new approach to testing equal accuracy," Journal of Econometrics, Elsevier, vol. 186(1), pages 160-177.
    2. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    3. Moscone, F. & Tosetti, Elisa, 2015. "Robust estimation under error cross section dependence," Economics Letters, Elsevier, vol. 133(C), pages 100-104.
    4. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
    5. Jushan Bai & Serena Ng, 2002. "Determining the Number of Factors in Approximate Factor Models," Econometrica, Econometric Society, vol. 70(1), pages 191-221, January.
    6. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    7. Kelejian, Harry H & Prucha, Ingmar R, 1998. "A Generalized Spatial Two-Stage Least Squares Procedure for Estimating a Spatial Autoregressive Model with Autoregressive Disturbances," The Journal of Real Estate Finance and Economics, Springer, vol. 17(1), pages 99-121, July.
    8. Kelejian, Harry H. & Prucha, Ingmar R., 2007. "HAC estimation in a spatial framework," Journal of Econometrics, Elsevier, vol. 140(1), pages 131-154, September.
    9. Moscone, Francesco & Tosetti, Elisa, 2012. "HAC estimation in spatial panels," Economics Letters, Elsevier, vol. 117(1), pages 60-65.
    10. Pesaran, M. Hashem & Tosetti, Elisa, 2011. "Large panels with common factors and spatial correlation," Journal of Econometrics, Elsevier, vol. 161(2), pages 182-202, April.
    11. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    12. T. S. Breusch & A. R. Pagan, 1980. "The Lagrange Multiplier Test and its Applications to Model Specification in Econometrics," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 47(1), pages 239-253.
    13. M. Hashem Pesaran & Aman Ullah & Takashi Yamagata, 2008. "A bias-adjusted LM test of error cross-section independence," Econometrics Journal, Royal Economic Society, vol. 11(1), pages 105-127, March.
    14. Vuong, Quang H, 1989. "Likelihood Ratio Tests for Model Selection and Non-nested Hypotheses," Econometrica, Econometric Society, vol. 57(2), pages 307-333, March.
    15. Andrews, Donald W K, 1991. "Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation," Econometrica, Econometric Society, vol. 59(3), pages 817-858, May.
    16. Alexander Chudik & M. Hashem Pesaran & Elisa Tosetti, 2011. "Weak and strong cross‐section dependence and estimation of large panels," Econometrics Journal, Royal Economic Society, vol. 14(1), pages 45-90, February.
    17. John C. Driscoll & Aart C. Kraay, 1998. "Consistent Covariance Matrix Estimation With Spatially Dependent Panel Data," The Review of Economics and Statistics, MIT Press, vol. 80(4), pages 549-560, November.
    18. Clark, Todd E. & West, Kenneth D., 2007. "Approximately normal tests for equal predictive accuracy in nested models," Journal of Econometrics, Elsevier, vol. 138(1), pages 291-311, May.
    19. Davies, Anthony & Lahiri, Kajal, 1995. "A new framework for analyzing survey forecasts using three-dimensional panel data," Journal of Econometrics, Elsevier, vol. 68(1), pages 205-227, July.
    20. Nigel Pain & Christine Lewis & Thai-Thanh Dang & Yosuke Jin & Pete Richardson, 2014. "OECD Forecasts During and After the Financial Crisis: A Post Mortem," OECD Economics Department Working Papers 1107, OECD Publishing.
    21. repec:hal:journl:peer-00796743 is not listed on IDEAS
    22. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    23. David Turner, 2017. "Designing fan charts for GDP growth forecasts to better reflect downturn risks," OECD Economics Department Working Papers 1428, OECD Publishing.
    24. Timmermann, Allan & Zhu, Yinchu, 2019. "Comparing Forecasting Performance with Panel Data," CEPR Discussion Papers 13746, C.E.P.R. Discussion Papers.
    25. Jushan Bai, 2003. "Inferential Theory for Factor Models of Large Dimensions," Econometrica, Econometric Society, vol. 71(1), pages 135-171, January.
    26. Jushan Bai & Serena Ng, 2006. "Confidence Intervals for Diffusion Index Forecasts and Inference for Factor-Augmented Regressions," Econometrica, Econometric Society, vol. 74(4), pages 1133-1150, July.
    27. Christine Lewis & Nigel Pain, 2014. "Lessons from OECD forecasts during and after the financial crisis," OECD Journal: Economic Studies, OECD Publishing, vol. 2014(1), pages 9-39.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Christophe BOUCHER & Wassim LE LANN & Stéphane MATTON & Sessi TOKPAVI, 2021. "Backtesting ESG Ratings," LEO Working Papers / DR LEO 2883, Orleans Economics Laboratory / Laboratoire d'Economie d'Orleans (LEO), University of Orleans.
    2. Ryan Greenaway-McGrevy & Kade Sorensen, 2021. "A spatial model averaging approach to measuring house prices," Journal of Spatial Econometrics, Springer, vol. 2(1), pages 1-32, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mu-Chun Wang, 2009. "Comparing the DSGE model with the factor model: an out-of-sample forecasting experiment," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 28(2), pages 167-182.
    2. Clark, Todd & McCracken, Michael, 2013. "Advances in Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1107-1201, Elsevier.
    3. Raffaella Giacomini & Barbara Rossi, 2013. "Forecasting in macroeconomics," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 17, pages 381-408, Edward Elgar Publishing.
    4. Firmin Doko Tchatoka & Qazi Haque, 2023. "On bootstrapping tests of equal forecast accuracy for nested models," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(7), pages 1844-1864, November.
    5. Jack Fosten, 2016. "Forecast evaluation with factor-augmented models," University of East Anglia School of Economics Working Paper Series 2016-05, School of Economics, University of East Anglia, Norwich, UK..
    6. Norman R. Swanson & Weiqi Xiong, 2018. "Big data analytics in economics: What have we learned so far, and where should we go from here?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 51(3), pages 695-746, August.
    7. Rossi, Barbara & Sekhposyan, Tatevik, 2011. "Understanding models' forecasting performance," Journal of Econometrics, Elsevier, vol. 164(1), pages 158-172, September.
    8. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    9. Mayer, Walter J. & Liu, Feng & Dang, Xin, 2017. "Improving the power of the Diebold–Mariano–West test for least squares predictions," International Journal of Forecasting, Elsevier, vol. 33(3), pages 618-626.
    10. West, Kenneth D., 2006. "Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 3, pages 99-134, Elsevier.
    11. Shintani, Mototsugu, 2005. "Nonlinear Forecasting Analysis Using Diffusion Indexes: An Application to Japan," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 37(3), pages 517-538, June.
    12. Mogliani, Matteo & Darné, Olivier & Pluyaud, Bertrand, 2017. "The new MIBA model: Real-time nowcasting of French GDP using the Banque de France's monthly business survey," Economic Modelling, Elsevier, vol. 64(C), pages 26-39.
    13. Vasilis Sarafidis & Tom Wansbeek, 2012. "Cross-Sectional Dependence in Panel Data Analysis," Econometric Reviews, Taylor & Francis Journals, vol. 31(5), pages 483-531, September.
    14. Kelly, Bryan & Pruitt, Seth, 2015. "The three-pass regression filter: A new approach to forecasting using many predictors," Journal of Econometrics, Elsevier, vol. 186(2), pages 294-316.
    15. Min Seong Kim, 2021. "Robust Inference for Diffusion-Index Forecasts with Cross-Sectionally Dependent Data," Working papers 2021-04, University of Connecticut, Department of Economics.
    16. Pincheira, Pablo & Hardy, Nicolas, 2022. "Correlation Based Tests of Predictability," MPRA Paper 112014, University Library of Munich, Germany.
    17. Akgun, Oguzhan & Pirotte, Alain & Urga, Giovanni, 2020. "Forecasting using heterogeneous panels with cross-sectional dependence," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1211-1227.
    18. Blasques, F. & Koopman, S.J. & Mallee, M. & Zhang, Z., 2016. "Weighted maximum likelihood for dynamic factor analysis and forecasting with mixed frequency data," Journal of Econometrics, Elsevier, vol. 193(2), pages 405-417.
    19. Jörg Breitung & Malte Knüppel, 2021. "How far can we forecast? Statistical tests of the predictive content," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 36(4), pages 369-392, June.
    20. Christophe Boucher & Wassim Le Lann & Stéphane Matton & Sessi Tokpavi, 2023. "Are ESG ratings informative to forecast idiosyncratic risk?," Working Papers hal-04140193, HAL.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2003.02803. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.