On the Usefulness of the Diebold-Mariano Test in the Selection of Prediction Models
In evaluating prediction models, many researchers flank comparative ex-ante prediction experiments by significance tests on accuracy improvement, such as the Diebold-Mariano test. We argue that basing the choice of prediction models on such significance tests is problematic, as this practice may favor the null model, usually a simple benchmark. We explore the validity of this argument by extensive Monte Carlo simulations with linear (ARMA) and nonlinear (SETAR) generating processes. For many parameter constellations, we find that utilization of additional significance tests in selecting the forecasting model fails to improve predictive accuracy.
|Date of creation:||Nov 2011|
|Date of revision:|
|Contact details of provider:|| Postal: Josefstädterstr. 39, A-1080 Vienna, Austria|
Phone: ++43 - (0)1 - 599 91 - 0
Fax: ++43 - (0)1 - 599 91 - 555
Web page: http://www.ihs.ac.at
More information through EDIRC
|Order Information:|| Postal: Institute for Advanced Studies - Library, Josefstädterstr. 39, A-1080 Vienna, Austria|
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Todd E. Clark & Michael McCracken, 1999.
"Tests of Equal Forecast Accuracy and Encompassing for Nested Models,"
Computing in Economics and Finance 1999
1241, Society for Computational Economics.
- Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
- Todd E. Clark & Michael W. McCracken, 1999. "Tests of equal forecast accuracy and encompassing for nested models," Research Working Paper 99-11, Federal Reserve Bank of Kansas City.
- Todd E. Clark & Michael W. McCracken, 2000. "Tests of Equal Forecast Accuracy and Encompassing for Nested Models," Econometric Society World Congress 2000 Contributed Papers 0319, Econometric Society.
- Costantini, Mauro & Kunst, Robert M., 2009.
"Combining Forecasts Based on Multiple Encompassing Tests in a Macroeconomic Core System,"
243, Institute for Advanced Studies.
- Mauro Costantini & Robert M. Kunst, 2011. "Combining forecasts based on multiple encompassing tests in a macroeconomic core system," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 30(6), pages 579-596, September.
- Inoue, Atsushi & Kilian, Lutz, 2003.
"On the Selection of Forecasting Models,"
CEPR Discussion Papers
3809, C.E.P.R. Discussion Papers.
- Bénédicte Vidaillet & V. D'Estaintot & P. Abécassis, 2005. "Introduction," Post-Print hal-00287137, HAL.
When requesting a correction, please mention this item's handle: RePEc:ihs:ihsesp:276. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Doris Szoncsitz)
If references are entirely missing, you can add them using this form.