This file is part of IDEAS, which uses RePEc data


[ Papers | Articles | Software | Books | Chapters | Authors | Institutions | JEL Classification | NEP reports | Search | New papers by email | Author registration | Rankings | Volunteers | FAQ | Blog | Help! ]

Tests of Conditional Predictive Ability

Author info | Abstract | Publisher info | Download info | Related research | Statistics
Author Info
Raffaella Giacomini (University of California, San Diego)
Halbert White (University of California, San Diego)
Abstract

We argue that the current framework for predictive ability testing (e.g., West, 1996) is not necessarily useful for real-time forecast selection, i.e., for assessing which of two competing forecasting methods will perform better in the future. We propose an alternative framework for out-of-sample comparison of predictive ability which delivers more practically relevant conclusions. Our approach is based on inference about conditional expectations of forecasts and forecast errors rather than the unconditional expectations that are the focus of the existing literature. We capture important determinants of forecast performance that are neglected in the existing literature by evaluating what we call the forecasting method (the model and the parameter estimation procedure), rather than just the forecasting model. Compared to previous approaches, our tests are valid under more general data assumptions (heterogeneity rather than stationarity) and estimation methods, and they can handle comparison of both nested and non-nested models, which is not currently possible. To illustrate the usefulness of the proposed tests, we compare the forecast performance of three leading parameter-reduction methods for macroeconomic forecasting using a large number of predictors: a sequential model selection approach, the "diffusion indexes" approach of Stock and Watson (2002), and the use of Bayesian shrinkage estimators.

Download Info
To download:

If you experience problems downloading a file, check if you have the proper application to view it first. Information about this may be contained in the File-Format links below. In case of further problems read the IDEAS help file. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://repositories.cdlib.org/cgi/viewcontent.cgi?article=1012&context=ucsdecon
File Format: application/pdf
File Function:
Download Restriction: no

Publisher Info
Paper provided by Department of Economics, UC San Diego in its series University of California at San Diego, Economics Working Paper Series with number 2003-09.

Download reference. The following formats are available: HTML, plain text, BibTeX, RIS (EndNote), ReDIF
Length:
Date of creation: 29 Sep 2004
Date of revision:
Handle: RePEc:cdl:ucsdec:2003-09

Note: oai:cdlib1.org:ucsdecon-1012
Contact details of provider:
Postal: 9500 Gilman Drive, La Jolla, CA 92093-0508
Phone: (858) 534-3383
Fax: (858) 534-7040
Web page: http://repositories.cdlib.org/ucsdecon/
More information through EDIRC

For technical questions regarding this item, or to correct its listing, contact: (Christopher F. Baum).

Related research
Keywords: Forecast evaluation asymptotic inference parameter-reduction methods

Other versions of this item:

This paper has been announced in the following NEP Reports: References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
  1. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November. [Downloadable!] (restricted)
    Other versions:
  2. Newey, Whitney K & West, Kenneth D, 1987. "A Simple, Positive Semi-definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix," Econometrica, Econometric Society, vol. 55(3), pages 703-08, May. [Downloadable!] (restricted)
    Other versions:
  3. Francis X. Diebold & Jose A. Lopez, 1996. "Forecast Evaluation and Combination," NBER Technical Working Papers 0192, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
    Other versions:
  4. Andrews, Donald W K, 1991. "Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation," Econometrica, Econometric Society, vol. 59(3), pages 817-58, May. [Downloadable!] (restricted)
    Other versions:
  5. Kevin D. Hoover & Stephen J. Perez, 1999. "Data mining reconsidered: encompassing and the general-to-specific approach to specification search," Econometrics Journal, Royal Economic Society, vol. 2(2), pages 167-191.
    Other versions:
  6. West, Kenneth D. & Edison, Hali J. & Cho, Dongchul, 1993. "A utility-based comparison of some models of exchange rate volatility," Journal of International Economics, Elsevier, vol. 35(1-2), pages 23-45, August. [Downloadable!] (restricted)
    Other versions:
  7. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
  8. White, Halbert & Domowitz, Ian, 1984. "Nonlinear Regression with Dependent Observations," Econometrica, Econometric Society, vol. 52(1), pages 143-61, January. [Downloadable!] (restricted)
  9. Mc Cracken, Michael W., 2000. "Robust out-of-sample inference," Journal of Econometrics, Elsevier, vol. 99(2), pages 195-223, December. [Downloadable!] (restricted)
  10. Harvey, David & Leybourne, Stephen & Newbold, Paul, 1997. "Testing the equality of prediction mean squared errors," International Journal of Forecasting, Elsevier, vol. 13(2), pages 281-291, June. [Downloadable!] (restricted)
  11. Bierens, Herman J, 1990. "A Consistent Conditional Moment Test of Functional Form," Econometrica, Econometric Society, vol. 58(6), pages 1443-58, November. [Downloadable!] (restricted)
  12. Fama, Eugene F & MacBeth, James D, 1973. "Risk, Return, and Equilibrium: Empirical Tests," Journal of Political Economy, University of Chicago Press, vol. 81(3), pages 607-36, May-June. [Downloadable!] (restricted)
  13. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-84, September. [Downloadable!] (restricted)
    Other versions:
  14. West, Kenneth D & McCracken, Michael W, 1998. "Regression-Based Tests of Predictive Ability," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 817-40, November.
    Other versions:
  15. Donald W. K. Andrews, 2002. "Higher-Order Improvements of a Computationally Attractive "k"-Step Bootstrap for Extremum Estimators," Econometrica, Econometric Society, vol. 70(1), pages 119-162, January. [Downloadable!] (restricted)
    Other versions:
  16. Diebold, Francis X & Mariano, Roberto S, 1995. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(3), pages 253-63, July.
    Other versions:
Full references

Cited by:
(explanations, Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.)

  1. Raffaella Giacomini & Ivana Komunjer, 2003. "Evaluation and Combination of Conditional Quantile Forecasts," Boston College Working Papers in Economics 571, Boston College Department of Economics. [Downloadable!]
    Other versions:
  2. Paul D. McNelis & Salih N. Neftci, 2006. "Renminbi Revaluation, Euro Appreciation and Chinese Markets: What Can We Learn From Data?," Working Papers 012006, Hong Kong Institute for Monetary Research. [Downloadable!]
  3. Todd E. Clark & Kenneth D. West, 2004. "Using out-of-sample mean squared prediction errors to test the Martingale difference hypothesis," Research Working Paper RWP 04-03, Federal Reserve Bank of Kansas City. [Downloadable!]
    Other versions:
  4. Hofmann, Boris, 2006. "Do monetary indicators (still) predict euro area inflation?," Discussion Paper Series 1: Economic Studies 2006,18, Deutsche Bundesbank, Research Centre. [Downloadable!]
  5. Todd E. Clark & Kenneth D. West, 2005. "Using Out-of-Sample Mean Squared Prediction Errors to Test the Martingale Difference," NBER Technical Working Papers 0305, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
  6. Gianni Amisano & Raffaella Giacomini, 2005. "Comparing Density Forecsts via Weighted Likelihood Ratio Tests," Working Papers ubs0504, University of Brescia, Department of Economics. [Downloadable!]
  7. Todd E. Clark & Kenneth D. West, 2005. "Approximately normal tests for equal predictive accuracy in nested models," Research Working Paper RWP 05-05, Federal Reserve Bank of Kansas City. [Downloadable!]
    Other versions:
  8. Isao Ishida, 2005. "Scanning Multivariate Conditional Densities with Probability Integral Transforms," CIRJE F-Series CIRJE-F-369, CIRJE, Faculty of Economics, University of Tokyo. [Downloadable!]
  9. Xiaohong Chen & Yanqin Fan, 2004. "Estimation and Model Selection of Semiparametric Copula-Based Multivariate Dynamic Models under Copula Misspecification," Working Papers 0419, Department of Economics, Vanderbilt University, revised Sep 2004. [Downloadable!]
Statistics
Access and download statistics

Did you know? Cannot find something on IDEAS? Encourage the publisher to index it! Instructions.

This page was last updated on 2008-8-30.

This information is provided to you by IDEAS at the Department of Economics, College of Liberal Arts and Sciences, University of Connecticut using RePEc data on a server sponsored by the Society for Economic Dynamics.