On the Usefulness of the Diebold-Mariano Test in the Selection of Prediction Models
In evaluating prediction models, many researchers flank comparative ex-ante prediction experiments by significance tests on accuracy improvement, such as the Diebold-Mariano test. We argue that basing the choice of prediction models on such significance tests is problematic, as this practice may favor the null model, usually a simple benchmark. We explore the validity of this argument by extensive Monte Carlo simulations with linear (ARMA) and nonlinear (SETAR) generating processes. For many parameter constellations, we find that utilization of additional significance tests in selecting the forecasting model fails to improve predictive accuracy.
|Date of creation:||Nov 2011|
|Date of revision:|
|Contact details of provider:|| Postal: |
Phone: ++43 - (0)1 - 599 91 - 0
Fax: ++43 - (0)1 - 599 91 - 555
Web page: http://www.ihs.ac.at
More information through EDIRC
|Order Information:|| Postal: Institute for Advanced Studies - Library, Stumpergasse 56, A-1060 Vienna, Austria|
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Costantini, Mauro & Kunst, Robert M., 2009.
"Combining Forecasts Based on Multiple Encompassing Tests in a Macroeconomic Core System,"
243, Institute for Advanced Studies.
- Mauro Costantini & Robert M. Kunst, 2011. "Combining forecasts based on multiple encompassing tests in a macroeconomic core system," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 30(6), pages 579-596, September.
- Todd E. Clark & Michael W. McCracken, 2000.
"Tests of Equal Forecast Accuracy and Encompassing for Nested Models,"
Econometric Society World Congress 2000 Contributed Papers
0319, Econometric Society.
- Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
- Todd E. Clark & Michael W. McCracken, 1999. "Tests of equal forecast accuracy and encompassing for nested models," Research Working Paper 99-11, Federal Reserve Bank of Kansas City.
- Todd E. Clark & Michael McCracken, 1999. "Tests of Equal Forecast Accuracy and Encompassing for Nested Models," Computing in Economics and Finance 1999 1241, Society for Computational Economics.
- repec:hal:journl:hal-00287137 is not listed on IDEAS
- Inoue, Atsushi & Kilian, Lutz, 2003.
"On the Selection of Forecasting Models,"
CEPR Discussion Papers
3809, C.E.P.R. Discussion Papers.
- Costantini, Mauro & Kunst, Robert M., 2011. "On the Usefulness of the Diebold-Mariano Test in the Selection of Prediction Models," Economics Series 276, Institute for Advanced Studies.
When requesting a correction, please mention this item's handle: RePEc:ihs:ihsesp:276. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Doris Szoncsitz)
If references are entirely missing, you can add them using this form.