IDEAS home Printed from https://ideas.repec.org/a/eee/ecolet/v256y2025ics0165176525004835.html

A new self-normalized forecast comparison test

Author

Listed:
  • Li, Haiqi
  • Zhang, Ni
  • Zhou, Jin

Abstract

This study develops a novel self-normalized Diebold–Mariano (DM) test for evaluating equal forecast accuracy. The proposed test offers several distinct advantages: it avoids bandwidth selection and bypasses direct estimation of long-run variances, both of which are typically required in conventional forecast accuracy testing approaches. Under relatively mild regularity conditions, we show that the asymptotic null distribution of the self-normalized DM test statistics is pivotal, with corresponding critical values tabulated through simulations. Comprehensive Monte Carlo simulations confirm that our self-normalized DM test has superior finite-sample performances compared to the original and the existing modified DM tests.

Suggested Citation

  • Li, Haiqi & Zhang, Ni & Zhou, Jin, 2025. "A new self-normalized forecast comparison test," Economics Letters, Elsevier, vol. 256(C).
    • Handle: RePEc:eee:ecolet:v:256:y:2025:i:c:s0165176525004835
    DOI: 10.1016/j.econlet.2025.112646
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0165176525004835
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.econlet.2025.112646?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. David I. Harvey & Stephen J. Leybourne & Yang Zu, 2025. "Testing for Equal Average Forecast Accuracy in Possibly Unstable Environments," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 43(3), pages 643-656, July.
    2. Lobato I. N., 2001. "Testing That a Dependent Process Is Uncorrelated," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1066-1076, September.
    3. Peter C.B. Phillips, 1987. "Multiple Regression with Integrated Time Series," Cowles Foundation Discussion Papers 852, Cowles Foundation for Research in Economics, Yale University.
    4. Xiaofeng Shao, 2010. "A self‐normalized approach to confidence interval construction in time series," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 72(3), pages 343-366, June.
    5. Amat, Christophe & Michalski, Tomasz & Stoltz, Gilles, 2018. "Fundamentals and exchange rate forecastability with simple machine learning methods," Journal of International Money and Finance, Elsevier, vol. 88(C), pages 1-24.
    6. David I. Harvey & Stephen J. Leybourne & Yang Zu, 2024. "Tests for equal forecast accuracy under heteroskedasticity," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 39(5), pages 850-869, August.
    7. Borup, Daniel & Schütte, Erik Christian Montes, 2022. "Asset pricing with data revisions," Journal of Financial Markets, Elsevier, vol. 59(PB).
    8. Zhou, Jin & Li, Haiqi & Zhong, Wanling, 2021. "A modified Diebold–Mariano test for equal forecast accuracy with clustered dependence," Economics Letters, Elsevier, vol. 207(C).
    9. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    10. Phillips, P C B, 1987. "Time Series Regression with a Unit Root," Econometrica, Econometric Society, vol. 55(2), pages 277-301, March.
    11. Andrews, Donald W K, 1991. "Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation," Econometrica, Econometric Society, vol. 59(3), pages 817-858, May.
    12. Andrews, Donald W K & Monahan, J Christopher, 1992. "An Improved Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimator," Econometrica, Econometric Society, vol. 60(4), pages 953-966, July.
    13. Nicholas M. Kiefer & Timothy J. Vogelsang, 2002. "Heteroskedasticity-Autocorrelation Robust Standard Errors Using The Bartlett Kernel Without Truncation," Econometrica, Econometric Society, vol. 70(5), pages 2093-2095, September.
    14. Khalaf, Lynda & Saunders, Charles J., 2017. "Monte Carlo forecast evaluation with persistent data," International Journal of Forecasting, Elsevier, vol. 33(1), pages 1-10.
    15. Philippe Goulet Coulombe & Maxime Leroux & Dalibor Stevanovic & Stéphane Surprenant, 2022. "How is machine learning useful for macroeconomic forecasting?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 920-964, August.
    16. Kiefer, Nicholas M. & Vogelsang, Timothy J., 2005. "A New Asymptotic Theory For Heteroskedasticity-Autocorrelation Robust Tests," Econometric Theory, Cambridge University Press, vol. 21(6), pages 1130-1164, December.
    17. Whitney Newey & Kenneth West, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 33(1), pages 125-132.
    18. Laura Coroneo & Fabrizio Iacone, 2020. "Comparing predictive accuracy in small samples using fixed‐smoothing asymptotics," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(4), pages 391-409, June.
    19. Zheng, Tingguo & Fan, Xinyue & Jin, Wei & Fang, Kuangnan, 2024. "Words or numbers? Macroeconomic nowcasting with textual and macroeconomic data," International Journal of Forecasting, Elsevier, vol. 40(2), pages 746-761.
    20. Harvey, David & Leybourne, Stephen & Newbold, Paul, 1997. "Testing the equality of prediction mean squared errors," International Journal of Forecasting, Elsevier, vol. 13(2), pages 281-291, June.
    21. Shao, Xiaofeng & Zhang, Xianyang, 2010. "Testing for Change Points in Time Series," Journal of the American Statistical Association, American Statistical Association, vol. 105(491), pages 1228-1240.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hong, Yongmiao & Linton, Oliver & McCabe, Brendan & Sun, Jiajing & Wang, Shouyang, 2024. "Kolmogorov–Smirnov type testing for structural breaks: A new adjusted-range based self-normalization approach," Journal of Econometrics, Elsevier, vol. 238(2).
    2. Hirukawa, Masayuki, 2023. "Robust Covariance Matrix Estimation in Time Series: A Review," Econometrics and Statistics, Elsevier, vol. 27(C), pages 36-61.
    3. Casini, Alessandro, 2023. "Theory of evolutionary spectra for heteroskedasticity and autocorrelation robust inference in possibly misspecified and nonstationary models," Journal of Econometrics, Elsevier, vol. 235(2), pages 372-392.
    4. repec:cam:camjip:2316 is not listed on IDEAS
    5. Casini, Alessandro & Perron, Pierre, 2024. "Prewhitened long-run variance estimation robust to nonstationarity," Journal of Econometrics, Elsevier, vol. 242(1).
    6. Sun, Jiajing & Hong, Yongmiao & Linton, Oliver & Zhao, Xiaolu, 2022. "Adjusted-range self-normalized confidence interval construction for censored dependent data," Economics Letters, Elsevier, vol. 220(C).
    7. Matei Demetrescu & Christoph Hanck & Robinson Kruse‐Becher, 2022. "Robust inference under time‐varying volatility: A real‐time evaluation of professional forecasters," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 1010-1030, August.
    8. Bunzel, Helle, 2006. "FIXED-b ASYMPTOTICS IN SINGLE-EQUATION COINTEGRATION MODELS WITH ENDOGENOUS REGRESSORS," Econometric Theory, Cambridge University Press, vol. 22(4), pages 743-755, August.
    9. Kruse, Robinson & Leschinski, Christian & Will, Michael, 2016. "Comparing Predictive Accuracy under Long Memory - With an Application to Volatility Forecasting," Hannover Economic Papers (HEP) dp-571, Leibniz Universität Hannover, Wirtschaftswissenschaftliche Fakultät.
    10. Christis Katsouris, 2023. "Limit Theory under Network Dependence and Nonstationarity," Papers 2308.01418, arXiv.org, revised Aug 2023.
    11. Helle Bunzel, 2004. "Fixed Bandwidth Asymptotics in Single Equation Models of Cointegration with an Application to Money Demand," Econometric Society 2004 North American Summer Meetings 219, Econometric Society.
    12. Federico Belotti & Alessandro Casini & Leopoldo Catania & Stefano Grassi & Pierre Perron, 2023. "Simultaneous bandwidths determination for DK-HAC estimators and long-run variance estimation in nonparametric settings," Econometric Reviews, Taylor & Francis Journals, vol. 42(3), pages 281-306, February.
    13. Demetrescu, Matei & Hanck, Christoph & Kruse-Becher, Robinson, 2026. "Robust Fixed-b Inference in the Presence of Time-Varying Volatility," Econometrics and Statistics, Elsevier, vol. 37(C), pages 154-173.
    14. Pötscher, Benedikt M. & Preinerstorfer, David, 2017. "Further Results on Size and Power of Heteroskedasticity and Autocorrelation Robust Tests, with an Application to Trend Testing," MPRA Paper 81053, University Library of Munich, Germany.
    15. Matei Demetrescu & Christoph Hanck & Robinson Kruse, 2016. "Fixed-b Inference in the Presence of Time-Varying Volatility," CREATES Research Papers 2016-01, Department of Economics and Business Economics, Aarhus University.
    16. Boudoukh, Jacob & Israel, Ronen & Richardson, Matthew, 2022. "Biases in long-horizon predictive regressions," Journal of Financial Economics, Elsevier, vol. 145(3), pages 937-969.
    17. Zhang, Jingsi & Jiang, Wenxin & Shao, Xiaofeng, 2013. "Bayesian model selection based on parameter estimates from subsamples," Statistics & Probability Letters, Elsevier, vol. 83(4), pages 979-986.
    18. Politis, D N, 2009. "Higher-Order Accurate, Positive Semi-definite Estimation of Large-Sample Covariance and Spectral Density Matrices," University of California at San Diego, Economics Working Paper Series qt66w826hz, Department of Economics, UC San Diego.
    19. Pötscher, Benedikt M. & Preinerstorfer, David, 2018. "Controlling the size of autocorrelation robust tests," Journal of Econometrics, Elsevier, vol. 207(2), pages 406-431.
    20. Håvard Hungnes, 2018. "Encompassing tests for evaluating multi-step system forecasts invariant to linear transformations," Discussion Papers 871, Statistics Norway, Research Department.
    21. Clark, Todd & McCracken, Michael, 2013. "Advances in Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1107-1201, Elsevier.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ecolet:v:256:y:2025:i:c:s0165176525004835. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ecolet .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.