IDEAS home Printed from https://ideas.repec.org/p/cqe/wpaper/10623.html
   My bibliography  Save this paper

Multi-horizon uniform superior predictive ability revisited: A size-exploiting and consistent test

Author

Listed:
  • Verena Monschang
  • Mark Trede
  • Bernd Wilfling

Abstract

Quaedvlieg (2021, Journal of Business & Economic Statistics) proposes a uniform Superior Predictive Ability (uSPA) test for comparing forecasts across multiple horizons. The procedure is based on a 'minimum Diebold-Mariano' test statistic, and asymptotic critical values are obtained via bootstrapping. We show, theoretically and via simulations, that Quaedvlieg's test is subject to massive size distortions. In this article, we establish several convergence results for the 'minimum Diebold- Mariano' statistic, revealing that appropriate asymptotic critical values are given by the quantiles of the standard normal distribution. The uSPA test modified this way (i) always keeps the nominal size, (ii) is size-exploiting along the boundary that separates the parameter subsets of the null and the alternative uSPA hypotheses, and (iii) is consistent. Based on the closed skew normal distribution, we present a procedure for approximating the power function and demonstrate the favorable finite-sample properties of our test. In an empirical replication, we find that Quaedvlieg's (2021) results on uSPA comparisons between direct and iterative forecasting methods are statistically inconclusive.

Suggested Citation

  • Verena Monschang & Mark Trede & Bernd Wilfling, 2023. "Multi-horizon uniform superior predictive ability revisited: A size-exploiting and consistent test," CQE Working Papers 10623, Center for Quantitative Economics (CQE), University of Muenster.
    • Handle: RePEc:cqe:wpaper:10623
    as

    Download full text from publisher

    File URL: https://www.wiwi.uni-muenster.de/cqe/sites/cqe/files/CQE_Paper/cqe_wp_106_2023.pdf
    File Function: Version of November 2023
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    2. Rogier Quaedvlieg, 2021. "Multi-Horizon Forecast Comparison," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(1), pages 40-53, January.
    3. Shorrocks, Anthony F, 1983. "Ranking Income Distributions," Economica, London School of Economics and Political Science, vol. 50(197), pages 3-17, February.
    4. Marcellino, Massimiliano & Stock, James H. & Watson, Mark W., 2006. "A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 499-526.
    5. Jack Fosten & Daniel Gutknecht, 2021. "Horizon confidence sets," Empirical Economics, Springer, vol. 61(2), pages 667-692, August.
    6. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    7. Peter R. Hansen & Asger Lunde & James M. Nason, 2011. "The Model Confidence Set," Econometrica, Econometric Society, vol. 79(2), pages 453-497, March.
    8. Harvey, David & Leybourne, Stephen & Newbold, Paul, 1997. "Testing the equality of prediction mean squared errors," International Journal of Forecasting, Elsevier, vol. 13(2), pages 281-291, June.
    9. Arellano-Valle, Reinaldo B. & Genton, Marc G., 2008. "On the exact distribution of the maximum of absolutely continuous dependent random variables," Statistics & Probability Letters, Elsevier, vol. 78(1), pages 27-35, January.
    10. de Jong, Robert M., 1997. "Central Limit Theorems for Dependent Heterogeneous Random Variables," Econometric Theory, Cambridge University Press, vol. 13(3), pages 353-367, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Terasvirta, Timo, 2006. "Forecasting economic variables with nonlinear models," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 8, pages 413-457, Elsevier.
    2. Wilms, Ines & Rombouts, Jeroen & Croux, Christophe, 2021. "Multivariate volatility forecasts for stock market indices," International Journal of Forecasting, Elsevier, vol. 37(2), pages 484-499.
    3. Hendry, David F. & Hubrich, Kirstin, 2006. "Forecasting economic aggregates by disaggregates," Working Paper Series 589, European Central Bank.
    4. Faust, Jon & Wright, Jonathan H., 2013. "Forecasting Inflation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 2-56, Elsevier.
    5. Ricardo P. Masini & Marcelo C. Medeiros & Eduardo F. Mendes, 2023. "Machine learning advances for time series forecasting," Journal of Economic Surveys, Wiley Blackwell, vol. 37(1), pages 76-111, February.
    6. Costas Milas & Phil Rothman, 2005. "Multivariate STAR Unemployment Rate Forecasts," Econometrics 0502010, University Library of Munich, Germany.
    7. Ferrara, Laurent & Marcellino, Massimiliano & Mogliani, Matteo, 2015. "Macroeconomic forecasting during the Great Recession: The return of non-linearity?," International Journal of Forecasting, Elsevier, vol. 31(3), pages 664-679.
    8. Erik Kole & Thijs Markwat & Anne Opschoor & Dick van Dijk, 2017. "Forecasting Value-at-Risk under Temporal and Portfolio Aggregation," Journal of Financial Econometrics, Oxford University Press, vol. 15(4), pages 649-677.
    9. Milas, Costas & Rothman, Philip, 2008. "Out-of-sample forecasting of unemployment rates with pooled STVECM forecasts," International Journal of Forecasting, Elsevier, vol. 24(1), pages 101-121.
    10. Karim Barhoumi & Olivier Darné & Laurent Ferrara, 2010. "Are disaggregate data useful for factor analysis in forecasting French GDP?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 29(1-2), pages 132-144.
    11. Nikolay Robinzonov & Gerhard Tutz & Torsten Hothorn, 2012. "Boosting techniques for nonlinear time series models," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 96(1), pages 99-122, January.
    12. Yaojie Zhang & Yudong Wang & Feng Ma & Yu Wei, 2022. "To jump or not to jump: momentum of jumps in crude oil price volatility prediction," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 8(1), pages 1-31, December.
    13. Andrew Martinez, 2017. "Testing for Differences in Path Forecast Accuracy: Forecast-Error Dynamics Matter," Working Papers (Old Series) 1717, Federal Reserve Bank of Cleveland.
    14. Cepni, Oguzhan & Clements, Michael P., 2024. "How local is the local inflation factor? Evidence from emerging European countries," International Journal of Forecasting, Elsevier, vol. 40(1), pages 160-183.
    15. Bumho Son & Yunyoung Lee & Seongwan Park & Jaewook Lee, 2023. "Forecasting global stock market volatility: The impact of volatility spillover index in spatial‐temporal graph‐based model," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(7), pages 1539-1559, November.
    16. Tarassow, Artur, 2019. "Forecasting U.S. money growth using economic uncertainty measures and regularisation techniques," International Journal of Forecasting, Elsevier, vol. 35(2), pages 443-457.
    17. Gaglianone, Wagner Piazza & Marins, Jaqueline Terra Moura, 2017. "Evaluation of exchange rate point and density forecasts: An application to Brazil," International Journal of Forecasting, Elsevier, vol. 33(3), pages 707-728.
    18. Mariano, Roberto S. & Preve, Daniel, 2012. "Statistical tests for multiple forecast comparison," Journal of Econometrics, Elsevier, vol. 169(1), pages 123-130.
    19. Daniel Buncic, 2012. "Understanding forecast failure of ESTAR models of real exchange rates," Empirical Economics, Springer, vol. 43(1), pages 399-426, August.
    20. Scott Brave & R. Andrew Butters & Alejandro Justiniano, 2016. "Forecasting Economic Activity with Mixed Frequency Bayesian VARs," Working Paper Series WP-2016-5, Federal Reserve Bank of Chicago.

    More about this item

    Keywords

    Forecast evaluation; Joint-hypothesis testing; Stochastic dominance; Closed skew normal distribution;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cqe:wpaper:10623. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Susanne Deckwitz (email available below). General contact details of provider: https://edirc.repec.org/data/cqmuede.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.