On the Usefulness of the Diebold-Mariano Test in the Selection of Prediction Models
In evaluating prediction models, many researchers flank comparative ex-ante prediction experiments by significance tests on accuracy improvement, such as the Diebold-Mariano test. We argue that basing the choice of prediction models on such significance tests is problematic, as this practice may favor the null model, usually a simple benchmark. We explore the validity of this argument by extensive Monte Carlo simulations with linear (ARMA) and nonlinear (SETAR) generating processes. For many parameter constellations, we find that utilization of additional significance tests in selecting the forecasting model fails to improve predictive accuracy.
|Date of creation:||Nov 2011|
|Contact details of provider:|| Postal: Josefstädterstr. 39, A-1080 Vienna, Austria|
Phone: ++43 - (0)1 - 599 91 - 0
Fax: ++43 - (0)1 - 599 91 - 555
Web page: http://www.ihs.ac.at
More information through EDIRC
|Order Information:|| Postal: Institute for Advanced Studies - Library, Josefstädterstr. 39, A-1080 Vienna, Austria|
References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Costantini, Mauro & Kunst, Robert M., 2009.
"Combining Forecasts Based on Multiple Encompassing Tests in a Macroeconomic Core System,"
243, Institute for Advanced Studies.
- Mauro Costantini & Robert M. Kunst, 2011. "Combining forecasts based on multiple encompassing tests in a macroeconomic core system," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 30(6), pages 579-596, September.
- Bénédicte Vidaillet & V. D'Estaintot & P. Abécassis, 2005. "Introduction," Post-Print hal-00287137, HAL.
- Clark, Todd E. & McCracken, Michael W., 2001.
"Tests of equal forecast accuracy and encompassing for nested models,"
Journal of Econometrics,
Elsevier, vol. 105(1), pages 85-110, November.
- Todd E. Clark & Michael W. McCracken, 1999. "Tests of equal forecast accuracy and encompassing for nested models," Research Working Paper 99-11, Federal Reserve Bank of Kansas City.
- Todd E. Clark & Michael McCracken, 1999. "Tests of Equal Forecast Accuracy and Encompassing for Nested Models," Computing in Economics and Finance 1999 1241, Society for Computational Economics.
- Todd E. Clark & Michael W. McCracken, 2000. "Tests of Equal Forecast Accuracy and Encompassing for Nested Models," Econometric Society World Congress 2000 Contributed Papers 0319, Econometric Society.
- Inoue, Atsushi & Kilian, Lutz, 2003.
"On the selection of forecasting models,"
Working Paper Series
0214, European Central Bank.
When requesting a correction, please mention this item's handle: RePEc:ihs:ihsesp:276. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Doris Szoncsitz)
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If references are entirely missing, you can add them using this form.
If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.
Please note that corrections may take a couple of weeks to filter through the various RePEc services.