On the Usefulness of the Diebold-Mariano Test in the Selection of Prediction Models
AbstractIn evaluating prediction models, many researchers flank comparative ex-ante prediction experiments by significance tests on accuracy improvement, such as the Diebold-Mariano test. We argue that basing the choice of prediction models on such significance tests is problematic, as this practice may favor the null model, usually a simple benchmark. We explore the validity of this argument by extensive Monte Carlo simulations with linear (ARMA) and nonlinear (SETAR) generating processes. For many parameter constellations, we find that utilization of additional significance tests in selecting the forecasting model fails to improve predictive accuracy.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoPaper provided by Institute for Advanced Studies in its series Economics Series with number 276.
Length: 18 pages
Date of creation: Nov 2011
Date of revision:
Postal: Institute for Advanced Studies - Library, Stumpergasse 56, A-1060 Vienna, Austria
Find related papers by JEL classification:
- C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models &bull Diffusion Processes
- C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
- C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Inoue, Atsushi & Kilian, Lutz, 2003.
"On the Selection of Forecasting Models,"
CEPR Discussion Papers
3809, C.E.P.R. Discussion Papers.
- Todd E. Clark & Michael McCracken, 1999.
"Tests of Equal Forecast Accuracy and Encompassing for Nested Models,"
Computing in Economics and Finance 1999
1241, Society for Computational Economics.
- Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, Elsevier, vol. 105(1), pages 85-110, November.
- Todd E. Clark & Michael W. McCracken, 1999. "Tests of equal forecast accuracy and encompassing for nested models," Research Working Paper, Federal Reserve Bank of Kansas City 99-11, Federal Reserve Bank of Kansas City.
- Todd E. Clark & Michael W. McCracken, 2000. "Tests of Equal Forecast Accuracy and Encompassing for Nested Models," Econometric Society World Congress 2000 Contributed Papers 0319, Econometric Society.
- Mauro Costantini & Robert M. Kunst, 2011.
"Combining forecasts based on multiple encompassing tests in a macroeconomic core system,"
Journal of Forecasting, John Wiley & Sons, Ltd.,
John Wiley & Sons, Ltd., vol. 30(6), pages 579-596, September.
- Costantini, Mauro & Kunst, Robert M., 2009. "Combining Forecasts Based on Multiple Encompassing Tests in a Macroeconomic Core System," Economics Series 243, Institute for Advanced Studies.
- Luis A. Gil-Alana & Juncal Cuñado & Guglielmo Maria Caporale, 2012.
"Modelling Long Run Trends and Cycles in Financial Time Series Data,"
Faculty Working Papers
13/12, School of Economics and Business Administration, University of Navarra.
- Guglielmo Maria Caporale & Juncal CuÃ±ado & Luis A. Gil-Alana, 2013. "Modelling long-run trends and cycles in financial time series data," Journal of Time Series Analysis, Wiley Blackwell, vol. 34(3), pages 405-421, 05.
- Guglielmo Maria Caporale & Juncal Cunado & Luis A. Gil-Alana, 2008. "Modelling Long-Run Trends and Cycles in Financial Time Series Data," CESifo Working Paper Series 2330, CESifo Group Munich.
- Anders Bredahl Kock & Timo Teräsvirta, 2011. "Forecasting Macroeconomic Variables using Neural Network Models and Three Automated Model Selection Techniques," CREATES Research Papers 2011-27, School of Economics and Management, University of Aarhus.
- Guglielmo Maria Caporale & Luis A. Gil-Alana, 2012.
"Persistence and Cycles in the US Federal Funds Rate,"
CESifo Working Paper Series
4035, CESifo Group Munich.
- Guglielmo Maria Caporale & Luis A. Gil-Alana, 2012. "Persistence and Cycles in the US Federal Funds Rate," Discussion Papers of DIW Berlin 1255, DIW Berlin, German Institute for Economic Research.
- FrÃ©dÃ©rique Bec & MÃ©lika Ben Salem, 2012.
"Inventory Investment and the Business Cycle : The usual Suspect,"
Working Papers, Centre de Recherche en Economie et Statistique
2012-09, Centre de Recherche en Economie et Statistique.
- Bec Frédérique & Salem Melika Ben, 2013. "Inventory investment and the business cycle: the usual suspect," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 17(3), pages 335-343, May.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Doris Szoncsitz).
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If references are entirely missing, you can add them using this form.
If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.
Please note that corrections may take a couple of weeks to filter through the various RePEc services.