IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Log in (now much improved!) to save this paper

Predective Density and Conditional Confidence Interval Accuracy Tests

  • Valentina Corradi

    ()

    (Queen Mary, University of London)

  • Norman Swanson

    ()

    (Rutgers University)

This paper outlines testing procedures for assessing the relative out-of-sample predictive accuracy of multiple conditional distribution models. The tests that are discussed are based on either the comparison of entire conditional distributions or the comparison of predictive confidence intervals. We also briefly survey existing related methods in the area of predictive density evaluation, including methods based on the probability integral transform and the Kullback-Leibler Information Criterion. The procedures proposed in this paper are similar in many ways to Andrews' (1997) conditional Kolmogorov test and to White's (2000) reality check. In particular, a predictive density test is outlined that involves comparing square (approximation) errors associated with models I, i=1,...,n, by constructing weighted averages over U of E[( F_{i}(u|Z^{t},theta _{i}^\dagger )-F_{0}(u|Z^{t}, theta _{0})) ^{2}] , where F_{0}(. |. ) and F_{i}(.|.)$ are true and model-i distributions, u belongs to U, and U is a possibly unbounded set on the real line. A conditional confidence interval version of this test is also outlined, and appropriate bootstrap procedures for obtaining critical values when predictions used in the formation of the test statistics are obtained via rolling and recursive estimation schemes are developed. An empirical illustration comparing alternative predictive models for U.S. inflation is given for the predictive confidence interval test.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.sas.rutgers.edu/virtual/snde/wp/2004-23.pdf
Download Restriction: no

Paper provided by Rutgers University, Department of Economics in its series Departmental Working Papers with number 200423.

as
in new window

Length: 20 pages
Date of creation: 17 Sep 2004
Date of revision:
Handle: RePEc:rut:rutres:200423
Contact details of provider: Postal:
New Jersey Hall - 75 Hamilton Street, New Brunswick, NJ 08901-1248

Phone: (732) 932-7363
Fax: (732) 932-7416
Web page: http://economics.rutgers.edu/

More information through EDIRC

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Donald W.K. Andrews, 1996. "A Conditional Kolmogorov Test," Cowles Foundation Discussion Papers 1111R, Cowles Foundation for Research in Economics, Yale University.
  2. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-83, November.
  3. Massimiliano Marcellino & James Stock & Mark Watson, 2005. "A Comparison of Direct and Iterated Multistep AR Methods for Forecasting Macroeconomic Time Series," Working Papers 285, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
  4. Frank Schorfheide, 2000. "Loss function-based evaluation of DSGE models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 15(6), pages 645-670.
  5. Hansen, Peter Reinhard, 2005. "A Test for Superior Predictive Ability," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 365-380, October.
  6. Francis X. Diebold & Robert S. Mariano, 1994. "Comparing Predictive Accuracy," NBER Technical Working Papers 0169, National Bureau of Economic Research, Inc.
  7. Yongsung Chang & Joao Gomes & Frank Schorfheide, 2002. "Learning by Doing as a Propagation Mechanism," Macroeconomics 0204002, EconWPA.
  8. Sílvia Gonçalves & Halbert White, 2002. "Maximum Likelihood and the Bootstrap for Nonlinear Dynamic Models," CIRANO Working Papers 2002s-41, CIRANO.
  9. Todd E. Clark & Michael W. McCracken, 1999. "Tests of equal forecast accuracy and encompassing for nested models," Research Working Paper 99-11, Federal Reserve Bank of Kansas City.
  10. Francis X. Diebold & Todd A. Gunther & Anthony S. Tay, 1997. "Evaluating density forecasts," Working Papers 97-6, Federal Reserve Bank of Philadelphia.
  11. Christian Bontemps & Nour Meddahi, 2012. "Testing distributional assumptions: A GMM aproach," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(6), pages 978-1012, 09.
  12. Francis X. Diebold & Jinyong Hahn & Anthony S. Tay, 1999. "Multivariate Density Forecast Evaluation And Calibration In Financial Risk Management: High-Frequency Returns On Foreign Exchange," The Review of Economics and Statistics, MIT Press, vol. 81(4), pages 661-673, November.
  13. Todd E. Clark & Michael W. McCracken, 2002. "Forecast-based model selection in the presence of structural breaks," Research Working Paper RWP 02-05, Federal Reserve Bank of Kansas City.
  14. Kenneth D. West & Michael W. McCracken, 1998. "Regression-Based Tests of Predictive Ability," NBER Technical Working Papers 0226, National Bureau of Economic Research, Inc.
  15. Hall, Peter & Horowitz, Joel L, 1996. "Bootstrap Critical Values for Tests Based on Generalized-Method-of-Moments Estimators," Econometrica, Econometric Society, vol. 64(4), pages 891-916, July.
  16. Linton, Oliver & Maasoumi, Esfandiar & Whang, Yoon-Jae, 2003. "Consistent Testing for Stochastic Dominance under General Sampling Schemes," SFB 373 Discussion Papers 2003,31, Humboldt University of Berlin, Interdisciplinary Research Project 373: Quantification and Simulation of Economic Processes.
  17. Donald W.K. Andrews, 1999. "Higher-Order Improvements of a Computationally Attractive-Step Bootstrap for Extremum Estimators," Cowles Foundation Discussion Papers 1230R, Cowles Foundation for Research in Economics, Yale University, revised Jan 2001.
  18. Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-62, November.
  19. Corradi, Valentina & Swanson, Norman R., 2005. "A Test For Comparing Multiple Misspecified Conditional Interval Models," Econometric Theory, Cambridge University Press, vol. 21(05), pages 991-1016, October.
  20. Christian Bontemps & Nour Meddahi, 2002. "Testing Normality: A GMM Approach," CIRANO Working Papers 2002s-63, CIRANO.
  21. Yongmiao Hong, 2005. "Nonparametric Specification Testing for Continuous-Time Models with Applications to Term Structure of Interest Rates," Review of Financial Studies, Society for Financial Studies, vol. 18(1), pages 37-84.
  22. Berkowitz, Jeremy, 2001. "Testing Density Forecasts, with Applications to Risk Management," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(4), pages 465-74, October.
  23. Giacomini, Raffaella, 2002. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests: Asymptotic and Bootstrap Methods," University of California at San Diego, Economics Working Paper Series qt59s2g5j5, Department of Economics, UC San Diego.
  24. West, K.D., 1994. "Asymptotic Inference About Predictive Ability," Working papers 9417, Wisconsin Madison - Social Systems.
  25. Valentina Corradi & Norman R. Swanson, 2003. "A Test for Comparing Multiple Misspecified Conditional Distributions," Departmental Working Papers 200314, Rutgers University, Department of Economics.
  26. Atsushi Inoue & Mototsugu Shintani, 2001. "Bootstrapping GMM Estimators for Time Series," Vanderbilt University Department of Economics Working Papers 0129, Vanderbilt University Department of Economics, revised Aug 2003.
  27. Peter Christoffersen & Jinyong Hahn & Atsushi Inoue, 2001. "Testing and Comparing Value-at-Risk Measures," CIRANO Working Papers 2001s-03, CIRANO.
  28. Corradi, Valentina, 1999. "Deciding Between I(0) And I(1) Via Flil-Based Bounds," Econometric Theory, Cambridge University Press, vol. 15(05), pages 643-663, October.
  29. Corradi, Valentina & Swanson, Norman R., 2002. "A consistent test for nonlinear out of sample predictive accuracy," Journal of Econometrics, Elsevier, vol. 110(2), pages 353-381, October.
  30. Peter Reinhard Hansen, 2001. "An Unbiased and Powerful Test for Superior Predictive Ability," Working Papers 2001-06, Brown University, Department of Economics.
  31. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
  32. Donald W.K. Andrews, 2002. "The Block-block Bootstrap: Improved Asymptotic Refinements," Cowles Foundation Discussion Papers 1370, Cowles Foundation for Research in Economics, Yale University.
  33. Valentina Corradi & Norman Swanson, 2004. "Bootstrap Procedures for Recursive Estimation Schemes With Applications to Forecast Model Selection," Departmental Working Papers 200418, Rutgers University, Department of Economics.
  34. Jushan Bai, 2003. "Testing Parametric Conditional Distributions of Dynamic Models," The Review of Economics and Statistics, MIT Press, vol. 85(3), pages 531-549, August.
  35. Allan Timmermann & M. Hashem Pesaran, 2003. "How Costly is it to Ignore Breaks when Forecasting the Direction of a Time Series?," CESifo Working Paper Series 875, CESifo Group Munich.
  36. McCracken, Michael W & Sapp, Stephen G, 2005. "Evaluating the Predictability of Exchange Rates Using Long-Horizon Regressions: Mind Your p's and q's!," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 37(3), pages 473-94, June.
  37. Valentina Corradi & Norman R. Swanson, 2003. "Bootstrap Conditional Distribution Tests In the Presence of Dynamic Misspecification," Departmental Working Papers 200311, Rutgers University, Department of Economics.
  38. Fernandez-Villaverde, Jesus & Francisco Rubio-Ramirez, Juan, 2004. "Comparing dynamic equilibrium models to data: a Bayesian approach," Journal of Econometrics, Elsevier, vol. 123(1), pages 153-187, November.
  39. Filippo Altissimo & Valentina Corradi, 2002. "Bounds for inference with nuisance parameters present only under the alternative," Econometrics Journal, Royal Economic Society, vol. 5(2), pages 494-519, 06.
  40. Wooldridge, Jeffrey M. & White, Halbert, 1988. "Some Invariance Principles and Central Limit Theorems for Dependent Heterogeneous Processes," Econometric Theory, Cambridge University Press, vol. 4(02), pages 210-230, August.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:rut:rutres:200423. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ()

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.