IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2008.08387.html
   My bibliography  Save this paper

A Novel Approach to Predictive Accuracy Testing in Nested Environments

Author

Listed:
  • Jean-Yves Pitarakis

Abstract

We introduce a new approach for comparing the predictive accuracy of two nested models that bypasses the difficulties caused by the degeneracy of the asymptotic variance of forecast error loss differentials used in the construction of commonly used predictive comparison statistics. Our approach continues to rely on the out of sample MSE loss differentials between the two competing models, leads to nuisance parameter free Gaussian asymptotics and is shown to remain valid under flexible assumptions that can accommodate heteroskedasticity and the presence of mixed predictors (e.g. stationary and local to unit root). A local power analysis also establishes its ability to detect departures from the null in both stationary and persistent settings. Simulations calibrated to common economic and financial applications indicate that our methods have strong power with good size control across commonly encountered sample sizes.

Suggested Citation

  • Jean-Yves Pitarakis, 2020. "A Novel Approach to Predictive Accuracy Testing in Nested Environments," Papers 2008.08387, arXiv.org.
  • Handle: RePEc:arx:papers:2008.08387
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2008.08387
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. McCracken, Michael W., 2007. "Asymptotics for out of sample tests of Granger causality," Journal of Econometrics, Elsevier, vol. 140(2), pages 719-752, October.
    2. Barbara Rossi, 2005. "Testing Long-Horizon Predictive Ability With High Persistence, And The Meese-Rogoff Puzzle," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 46(1), pages 61-92, February.
    3. Susanne M. Schennach & Daniel Wilhelm, 2017. "A Simple Parametric Model Selection Test," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(520), pages 1663-1674, October.
    4. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    5. Francis X. Diebold, 2015. "Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold-Mariano Tests," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 33(1), pages 1-1, January.
    6. Clark, Todd E. & West, Kenneth D., 2007. "Approximately normal tests for equal predictive accuracy in nested models," Journal of Econometrics, Elsevier, vol. 138(1), pages 291-311, May.
    7. Deng, Ai & Perron, Pierre, 2008. "The Limit Distribution Of The Cusum Of Squares Test Under General Mixing Conditions," Econometric Theory, Cambridge University Press, vol. 24(3), pages 809-822, June.
    8. Avdis, Efstathios & Wachter, Jessica A., 2017. "Maximum likelihood estimation of the equity premium," Journal of Financial Economics, Elsevier, vol. 125(3), pages 589-609.
    9. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
    10. Xiaoxia Shi, 2015. "A nondegenerate Vuong test," Quantitative Economics, Econometric Society, vol. 6(1), pages 85-121, March.
    11. Giraitis, Liudas & Kokoszka, Piotr & Leipus, Remigijus, 2000. "Stationary Arch Models: Dependence Structure And Central Limit Theorem," Econometric Theory, Cambridge University Press, vol. 16(1), pages 3-22, February.
    12. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    13. West, Kenneth D & McCracken, Michael W, 1998. "Regression-Based Tests of Predictive Ability," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 817-840, November.
    14. Deng, Ai & Perron, Pierre, 2008. "A non-local perspective on the power properties of the CUSUM and CUSUM of squares tests for structural change," Journal of Econometrics, Elsevier, vol. 142(1), pages 212-240, January.
    15. Molodtsova, Tanya & Papell, David H., 2009. "Out-of-sample exchange rate predictability with Taylor rule fundamentals," Journal of International Economics, Elsevier, vol. 77(2), pages 167-180, April.
    16. Berkes, István & Hörmann, Siegfried & Horváth, Lajos, 2008. "The functional central limit theorem for a family of GARCH observations with applications," Statistics & Probability Letters, Elsevier, vol. 78(16), pages 2725-2730, November.
    17. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    18. Peter Reinhard Hansen & Allan Timmermann, 2015. "Equivalence Between Out‐of‐Sample Forecast Comparisons and Wald Statistics," Econometrica, Econometric Society, vol. 83, pages 2485-2505, November.
    19. Meese, Richard A. & Rogoff, Kenneth, 1983. "Empirical exchange rate models of the seventies : Do they fit out of sample?," Journal of International Economics, Elsevier, vol. 14(1-2), pages 3-24, February.
    20. Berenguer-Rico, Vanessa & Nielsen, Bent, 2020. "Cumulated Sum Of Squares Statistics For Nonlinear And Nonstationary Regressions," Econometric Theory, Cambridge University Press, vol. 36(1), pages 1-47, February.
    21. Todd Clark & Michael McCracken, 2005. "Evaluating Direct Multistep Forecasts," Econometric Reviews, Taylor & Francis Journals, vol. 24(4), pages 369-404.
    22. Vuong, Quang H, 1989. "Likelihood Ratio Tests for Model Selection and Non-nested Hypotheses," Econometrica, Econometric Society, vol. 57(2), pages 307-333, March.
    23. Ferson, Wayne & Nallareddy, Suresh & Xie, Biqin, 2013. "The “out-of-sample” performance of long run risk models," Journal of Financial Economics, Elsevier, vol. 107(3), pages 537-556.
    24. Jianqing Fan & Yuan Liao & Jiawei Yao, 2015. "Power Enhancement in High‐Dimensional Cross‐Sectional Tests," Econometrica, Econometric Society, vol. 83(4), pages 1497-1541, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pitarakis, Jean-Yves & Gonzalo, Jesús, 2020. "Out of sample predictability in predictive regressions with many predictor candidates," UC3M Working papers. Economics 31554, Universidad Carlos III de Madrid. Departamento de Economía.
    2. Clark, Todd & McCracken, Michael, 2013. "Advances in Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1107-1201, Elsevier.
    3. Barbara Rossi & Atsushi Inoue, 2012. "Out-of-Sample Forecast Tests Robust to the Choice of Window Size," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(3), pages 432-453, April.
    4. Ince, Onur, 2014. "Forecasting exchange rates out-of-sample with panel methods and real-time data," Journal of International Money and Finance, Elsevier, vol. 43(C), pages 1-18.
    5. Clark, Todd E. & McCracken, Michael W., 2009. "Tests of Equal Predictive Ability With Real-Time Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(4), pages 441-454.
    6. Jamali, Ibrahim & Yamani, Ehab, 2019. "Out-of-sample exchange rate predictability in emerging markets: Fundamentals versus technical analysis," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 61(C), pages 241-263.
    7. Nikolsko-Rzhevskyy, Alex & Prodan, Ruxandra, 2012. "Markov switching and exchange rate predictability," International Journal of Forecasting, Elsevier, vol. 28(2), pages 353-365.
    8. Granziera, Eleonora & Hubrich, Kirstin & Moon, Hyungsik Roger, 2014. "A predictability test for a small number of nested models," Journal of Econometrics, Elsevier, vol. 182(1), pages 174-185.
    9. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    10. Rudan Wang & Bruce Morley & Javier Ordóñez, 2016. "The Taylor Rule, Wealth Effects and the Exchange Rate," Review of International Economics, Wiley Blackwell, vol. 24(2), pages 282-301, May.
    11. Brooks, Chris & Burke, Simon P. & Stanescu, Silvia, 2016. "Finite sample weighting of recursive forecast errors," International Journal of Forecasting, Elsevier, vol. 32(2), pages 458-474.
    12. Raffaella Giacomini & Barbara Rossi, 2013. "Forecasting in macroeconomics," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 17, pages 381-408, Edward Elgar Publishing.
    13. Mogliani, Matteo & Darné, Olivier & Pluyaud, Bertrand, 2017. "The new MIBA model: Real-time nowcasting of French GDP using the Banque de France's monthly business survey," Economic Modelling, Elsevier, vol. 64(C), pages 26-39.
    14. Pincheira, Pablo, 2017. "A Power Booster Factor for Out-of-Sample Tests of Predictability," MPRA Paper 77027, University Library of Munich, Germany.
    15. Ferrara, Laurent & Marcellino, Massimiliano & Mogliani, Matteo, 2015. "Macroeconomic forecasting during the Great Recession: The return of non-linearity?," International Journal of Forecasting, Elsevier, vol. 31(3), pages 664-679.
    16. Ahmed, Shamim & Liu, Xiaoquan & Valente, Giorgio, 2016. "Can currency-based risk factors help forecast exchange rates?," International Journal of Forecasting, Elsevier, vol. 32(1), pages 75-97.
    17. Rossi, Barbara & Sekhposyan, Tatevik, 2011. "Understanding models' forecasting performance," Journal of Econometrics, Elsevier, vol. 164(1), pages 158-172, September.
    18. West, Kenneth D., 2006. "Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 3, pages 99-134, Elsevier.
    19. Todd E. Clark & Michael W. Mccracken, 2014. "Tests Of Equal Forecast Accuracy For Overlapping Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 29(3), pages 415-430, April.
    20. Firmin Doko Tchatoka & Qazi Haque, 2020. "On bootstrapping tests of equal forecast accuracy for nested models," Economics Discussion / Working Papers 20-06, The University of Western Australia, Department of Economics.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2008.08387. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (arXiv administrators). General contact details of provider: http://arxiv.org/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.