On the Usefulness of the Diebold-Mariano Test in the Selection of Prediction Models
AbstractIn evaluating prediction models, many researchers flank comparative ex-ante prediction experiments by significance tests on accuracy improvement, such as the Diebold-Mariano test. We argue that basing the choice of prediction models on such significance tests is problematic, as this practice may favor the null model, usually a simple benchmark. We explore the validity of this argument by extensive Monte Carlo simulations with linear (ARMA) and nonlinear (SETAR) generating processes. For many parameter constellations, we find that utilization of additional significance tests in selecting the forecasting model fails to improve predictive accuracy.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoPaper provided by Institute for Advanced Studies in its series Economics Series with number 276.
Length: 18 pages
Date of creation: Nov 2011
Date of revision:
Postal: Institute for Advanced Studies - Library, Stumpergasse 56, A-1060 Vienna, Austria
Find related papers by JEL classification:
- C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models &bull Diffusion Processes
- C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
- C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Inoue, Atsushi & Kilian, Lutz, 2006.
"On the selection of forecasting models,"
Journal of Econometrics,
Elsevier, vol. 130(2), pages 273-306, February.
- Todd E. Clark & Michael W. McCracken, 1999.
"Tests of equal forecast accuracy and encompassing for nested models,"
Research Working Paper
99-11, Federal Reserve Bank of Kansas City.
- Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
- Todd E. Clark & Michael McCracken, 1999. "Tests of Equal Forecast Accuracy and Encompassing for Nested Models," Computing in Economics and Finance 1999 1241, Society for Computational Economics.
- Todd E. Clark & Michael W. McCracken, 2000. "Tests of Equal Forecast Accuracy and Encompassing for Nested Models," Econometric Society World Congress 2000 Contributed Papers 0319, Econometric Society.
- Costantini, Mauro & Kunst, Robert M., 2009.
"Combining Forecasts Based on Multiple Encompassing Tests in a Macroeconomic Core System,"
243, Institute for Advanced Studies.
- Mauro Costantini & Robert M. Kunst, 2011. "Combining forecasts based on multiple encompassing tests in a macroeconomic core system," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 30(6), pages 579-596, September.
- Bec Frédérique & Salem Melika Ben, 2013.
"Inventory investment and the business cycle: the usual suspect,"
Studies in Nonlinear Dynamics & Econometrics,
De Gruyter, vol. 17(3), pages 335-343, May.
- Frédérique Bec & Mélika Ben Salem, 2012. "Inventory Investment and the Business Cycle : The usual Suspect," Working Papers 2012-09, Centre de Recherche en Economie et Statistique.
- Guglielmo Maria Caporale & Luis A. Gil-Alana, 2012.
"Persistence and Cycles in the US Federal Funds Rate,"
CESifo Working Paper Series
4035, CESifo Group Munich.
- Guglielmo Maria Caporale & Luis A. Gil-Alana, 2012. "Persistence and Cycles in the US Federal Funds Rate," Discussion Papers of DIW Berlin 1255, DIW Berlin, German Institute for Economic Research.
- Guglielmo Maria Caporale & Juncal Cunado & Luis A. Gil-Alana, 2008.
"Modelling Long-Run Trends and Cycles in Financial Time Series Data,"
CESifo Working Paper Series
2330, CESifo Group Munich.
- Guglielmo Maria Caporale & Juncal Cuñado & Luis A. Gil-Alana, 2013. "Modelling long-run trends and cycles in financial time series data," Journal of Time Series Analysis, Wiley Blackwell, vol. 34(3), pages 405-421, 05.
- Luis A. Gil-Alana & Juncal Cuñado & Guglielmo Maria Caporale, 2012. "Modelling Long Run Trends and Cycles in Financial Time Series Data," Faculty Working Papers 13/12, School of Economics and Business Administration, University of Navarra.
- Anders Bredahl Kock & Timo Teräsvirta, 2011. "Forecasting Macroeconomic Variables using Neural Network Models and Three Automated Model Selection Techniques," CREATES Research Papers 2011-27, School of Economics and Management, University of Aarhus.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Doris Szoncsitz).
If references are entirely missing, you can add them using this form.