On the Usefulness of the Diebold-Mariano Test in the Selection of Prediction Models
In evaluating prediction models, many researchers flank comparative ex-ante prediction experiments by significance tests on accuracy improvement, such as the Diebold-Mariano test. We argue that basing the choice of prediction models on such significance tests is problematic, as this practice may favor the null model, usually a simple benchmark. We explore the validity of this argument by extensive Monte Carlo simulations with linear (ARMA) and nonlinear (SETAR) generating processes. For many parameter constellations, we find that utilization of additional significance tests in selecting the forecasting model fails to improve predictive accuracy.
|Date of creation:||Nov 2011|
|Contact details of provider:|| Postal: Josefstädterstr. 39, A-1080 Vienna, Austria|
Phone: ++43 - (0)1 - 599 91 - 0
Fax: ++43 - (0)1 - 599 91 - 555
Web page: http://www.ihs.ac.at
More information through EDIRC
|Order Information:|| Postal: Institute for Advanced Studies - Library, Josefstädterstr. 39, A-1080 Vienna, Austria|
References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Bénédicte Vidaillet & V. D'Estaintot & P. Abécassis, 2005. "Introduction," Post-Print hal-00287137, HAL.
- Mauro Costantini & Robert M. Kunst, 2011.
"Combining forecasts based on multiple encompassing tests in a macroeconomic core system,"
Journal of Forecasting,
John Wiley & Sons, Ltd., vol. 30(6), pages 579-596, September.
- Costantini, Mauro & Kunst, Robert M., 2009. "Combining Forecasts Based on Multiple Encompassing Tests in a Macroeconomic Core System," Economics Series 243, Institute for Advanced Studies.
- Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
- Todd E. Clark & Michael W. McCracken, 1999. "Tests of equal forecast accuracy and encompassing for nested models," Research Working Paper 99-11, Federal Reserve Bank of Kansas City.
- Todd E. Clark & Michael McCracken, 1999. "Tests of Equal Forecast Accuracy and Encompassing for Nested Models," Computing in Economics and Finance 1999 1241, Society for Computational Economics.
- Todd E. Clark & Michael W. McCracken, 2000. "Tests of Equal Forecast Accuracy and Encompassing for Nested Models," Econometric Society World Congress 2000 Contributed Papers 0319, Econometric Society.
- Inoue, Atsushi & Kilian, Lutz, 2006. "On the selection of forecasting models," Journal of Econometrics, Elsevier, vol. 130(2), pages 273-306, February.
- Inoue, Atsushi & Kilian, Lutz, 2003. "On the Selection of Forecasting Models," CEPR Discussion Papers 3809, C.E.P.R. Discussion Papers.
- Inoue, Atsushi & Kilian, Lutz, 2003. "On the selection of forecasting models," Working Paper Series 0214, European Central Bank.