IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Login to save this article or follow this journal

Predictive density and conditional confidence interval accuracy tests

  • Corradi, Valentina
  • Swanson, Norman R.

This paper outlines testing procedures for assessing the relative out-of-sample predictive accuracy of multiple conditional distribution models. The tests that are discussed are based on either the comparison of entire conditional distributions or the comparison of predictive confidence intervals. We also briefly survey existing related methods in the area of predictive density evaluation, including methods based on the probability integral transform and the Kullback-Leibler Information Criterion. The procedures proposed in this paper are similar in many ways to Andrews' (1997) conditional Kolmogorov test and to White's (2000) reality check. In particular, a predictive density test is outlined that involves comparing square (approximation) errors associated with models I, i=1,...,n, by constructing weighted averages over U of E[( F_{i}(u|Z^{t},theta _{i}^\dagger )-F_{0}(u|Z^{t}, theta _{0})) ^{2}] , where F_{0}(. |. ) and F_{i}(.|.)$ are true and model-i distributions, u belongs to U, and U is a possibly unbounded set on the real line. A conditional confidence interval version of this test is also outlined, and appropriate bootstrap procedures for obtaining critical values when predictions used in the formation of the test statistics are obtained via rolling and recursive estimation schemes are developed. An empirical illustration comparing alternative predictive models for U.S. inflation is given for the predictive confidence interval test.

(This abstract was borrowed from another version of this item.)

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.sciencedirect.com/science/article/B6VC0-4H2FY99-3/2/8afd56476baebd2ef768d6ce99adfef7
Download Restriction: Full text for ScienceDirect subscribers only

As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

Article provided by Elsevier in its journal Journal of Econometrics.

Volume (Year): 135 (2006)
Issue (Month): 1-2 ()
Pages: 187-228

as
in new window

Handle: RePEc:eee:econom:v:135:y:2006:i:1-2:p:187-228
Contact details of provider: Web page: http://www.elsevier.com/locate/jeconom

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Yongsung Chang & Joao F. Gomes & Frank Schorfheide, 2002. "Learning-by-Doing as a Propagation Mechanism," American Economic Review, American Economic Association, vol. 92(5), pages 1498-1520, December.
  2. Donald W.K. Andrews, 1999. "Higher-Order Improvements of a Computationally Attractive-Step Bootstrap for Extremum Estimators," Cowles Foundation Discussion Papers 1230, Cowles Foundation for Research in Economics, Yale University.
  3. Wooldridge, Jeffrey M. & White, Halbert, 1988. "Some Invariance Principles and Central Limit Theorems for Dependent Heterogeneous Processes," Econometric Theory, Cambridge University Press, vol. 4(02), pages 210-230, August.
  4. Donald W. K. Andrews, 2004. "the Block-Block Bootstrap: Improved Asymptotic Refinements," Econometrica, Econometric Society, vol. 72(3), pages 673-700, 05.
  5. Oliver Linton & Esfandiar Maasoumi & Yoon-Jae Whang, 2005. "Consistent Testing for Stochastic Dominance under General Sampling Schemes," Review of Economic Studies, Oxford University Press, vol. 72(3), pages 735-765.
  6. Sílvia Gonçalves & Halbert White, 2002. "Maximum Likelihood and the Bootstrap for Nonlinear Dynamic Models," CIRANO Working Papers 2002s-41, CIRANO.
  7. Valentina Corradi & Norman R. Swanson, 2003. "Bootstrap Conditional Distribution Tests In the Presence of Dynamic Misspecification," Departmental Working Papers 200311, Rutgers University, Department of Economics.
  8. Christoffersen, Peter & Hahn, Jinyong & Inoue, Atsushi, 2001. "Testing and comparing Value-at-Risk measures," Journal of Empirical Finance, Elsevier, vol. 8(3), pages 325-342, July.
  9. Marcellino, Massimiliano & Stock, James H. & Watson, Mark W., 2006. "A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 499-526.
  10. Jushan Bai, 2003. "Testing Parametric Conditional Distributions of Dynamic Models," The Review of Economics and Statistics, MIT Press, vol. 85(3), pages 531-549, August.
  11. Corradi, Valentina & Swanson, Norman R., 2005. "A Test For Comparing Multiple Misspecified Conditional Interval Models," Econometric Theory, Cambridge University Press, vol. 21(05), pages 991-1016, October.
  12. Hansen, Peter Reinhard, 2005. "A Test for Superior Predictive Ability," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 365-380, October.
  13. Francis X. Diebold & Todd A. Gunther & Anthony S. Tay, 1997. "Evaluating density forecasts," Working Papers 97-6, Federal Reserve Bank of Philadelphia.
  14. Pesaran, H.M. & Timmermann, A., 2003. "How Costly is it to Ignore Breaks when Forecasting the Direction of a Time Series?," Cambridge Working Papers in Economics 0306, Faculty of Economics, University of Cambridge.
  15. N. Meddahi & C. Bontemps, 2004. "Testing Distributional Assumptions: A GMM Approach," Econometric Society 2004 North American Winter Meetings 487, Econometric Society.
  16. Berkowitz, Jeremy, 2001. "Testing Density Forecasts, with Applications to Risk Management," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(4), pages 465-74, October.
  17. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-83, November.
  18. repec:att:wimass:9710 is not listed on IDEAS
  19. Francis X. Diebold & Robert S. Mariano, 1994. "Comparing Predictive Accuracy," NBER Technical Working Papers 0169, National Bureau of Economic Research, Inc.
  20. Yongmiao Hong, 2005. "Nonparametric Specification Testing for Continuous-Time Models with Applications to Term Structure of Interest Rates," Review of Financial Studies, Society for Financial Studies, vol. 18(1), pages 37-84.
  21. West, Kenneth D & McCracken, Michael W, 1998. "Regression-Based Tests of Predictive Ability," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 817-40, November.
  22. Corradi, V. & Swanson, N.R., 2000. "A Consistent Test for Nonlinear Out of Sample Predictive Accuracy," Discussion Papers 0012, Exeter University, Department of Economics.
  23. Corradi, Valentina, 1999. "Deciding Between I(0) And I(1) Via Flil-Based Bounds," Econometric Theory, Cambridge University Press, vol. 15(05), pages 643-663, October.
  24. Todd E. Clark & Michael McCracken, 1999. "Tests of Equal Forecast Accuracy and Encompassing for Nested Models," Computing in Economics and Finance 1999 1241, Society for Computational Economics.
  25. Donald W. K. Andrews, 1997. "A Conditional Kolmogorov Test," Econometrica, Econometric Society, vol. 65(5), pages 1097-1128, September.
  26. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-84, September.
  27. Inoue, Atsushi & Shintani, Mototsugu, 2006. "Bootstrapping GMM estimators for time series," Journal of Econometrics, Elsevier, vol. 133(2), pages 531-555, August.
  28. Todd E. Clark & Michael W. McCracken, 2002. "Forecast-based model selection in the presence of structural breaks," Research Working Paper RWP 02-05, Federal Reserve Bank of Kansas City.
  29. Valentina Corradi & Norman R. Swanson, 2003. "A Test for Comparing Multiple Misspecified Conditional Distributions," Departmental Working Papers 200314, Rutgers University, Department of Economics.
  30. BONTEMPS, Christian & MEDDAHI, Nour, 2002. "Testing Normality : A GMM Approach," Cahiers de recherche 2002-14, Universite de Montreal, Departement de sciences economiques.
  31. McCracken, Michael W & Sapp, Stephen G, 2005. "Evaluating the Predictability of Exchange Rates Using Long-Horizon Regressions: Mind Your p's and q's!," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 37(3), pages 473-94, June.
  32. Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-62, November.
  33. Hall, Peter & Horowitz, Joel L, 1996. "Bootstrap Critical Values for Tests Based on Generalized-Method-of-Moments Estimators," Econometrica, Econometric Society, vol. 64(4), pages 891-916, July.
  34. Francis X. Diebold & Jinyong Hahn & Anthony S. Tay, 1999. "Multivariate Density Forecast Evaluation And Calibration In Financial Risk Management: High-Frequency Returns On Foreign Exchange," The Review of Economics and Statistics, MIT Press, vol. 81(4), pages 661-673, November.
  35. Fernandez-Villaverde, Jesus & Francisco Rubio-Ramirez, Juan, 2004. "Comparing dynamic equilibrium models to data: a Bayesian approach," Journal of Econometrics, Elsevier, vol. 123(1), pages 153-187, November.
  36. Frank Schorfheide, 2000. "Loss function-based evaluation of DSGE models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 15(6), pages 645-670.
  37. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
  38. Peter Reinhard Hansen, 2001. "An Unbiased and Powerful Test for Superior Predictive Ability," Working Papers 2001-06, Brown University, Department of Economics.
  39. Valentina Corradi & Norman Swanson, 2004. "Bootstrap Procedures for Recursive Estimation Schemes With Applications to Forecast Model Selection," Departmental Working Papers 200418, Rutgers University, Department of Economics.
  40. Giacomini, Raffaella, 2002. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests: Asymptotic and Bootstrap Methods," University of California at San Diego, Economics Working Paper Series qt59s2g5j5, Department of Economics, UC San Diego.
  41. Filippo Altissimo & Valentina Corradi, 2002. "Bounds for inference with nuisance parameters present only under the alternative," Econometrics Journal, Royal Economic Society, vol. 5(2), pages 494-519, 06.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:135:y:2006:i:1-2:p:187-228. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.