IDEAS home Printed from https://ideas.repec.org/p/rut/rutres/200314.html
   My bibliography  Save this paper

A Test for Comparing Multiple Misspecified Conditional Distributions

Author

Listed:
  • Valentina Corradi

    () (Queen Mary, University of London)

  • Norman R. Swanson

    () (Rutgers University)

Abstract

This paper introduces a conditional Kolmogorov test, in the spirit of Andrews (1997), that allows for comparison of multiple misspecifed conditional distribution models, for the case of dependent observations. A conditional confidence interval version of the test is also discussed. Model accuracy is measured using a distributional analog of mean square error, in which the squared (approximation) error associated with a given model, say model i; is measured in terms of the average over U of E((Fi(u|Zt,Theta-t-plus)-Fo(u|Zt,Theta-o)))^2; where U is a possibly unbounded set on the real line, Zt is the conditioning information set, Fi is the distribution function of a particular candidate model, and F0 is the true (unkown) distribution function. When comparing more than two models, a “benchmark” model is specified, and the test is constructed along the lines of the “reality check” of White (2000). Valid asymptotic critical values are obtained via a version of the block bootstrap which properly captures the e®ect of parameter estimation error. The results of a small Monte Carlo experiment indicate that the conditional confidence interval version of the test has reasonable finite sample properties even for samples with as few as 60 observations.

Suggested Citation

  • Valentina Corradi & Norman R. Swanson, 2003. "A Test for Comparing Multiple Misspecified Conditional Distributions," Departmental Working Papers 200314, Rutgers University, Department of Economics.
  • Handle: RePEc:rut:rutres:200314
    as

    Download full text from publisher

    File URL: http://www.sas.rutgers.edu/virtual/snde/wp/2003-14.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. Francis X. Diebold & Todd A. Gunther & Anthony S. Tay, "undated". "Evaluating Density Forecasts," CARESS Working Papres 97-18, University of Pennsylvania Center for Analytic Research and Economics in the Social Sciences.
    2. Gon alves, S lvia & White, Halbert, 2002. "The Bootstrap Of The Mean For Dependent Heterogeneous Arrays," Econometric Theory, Cambridge University Press, vol. 18(06), pages 1367-1384, December.
    3. Oliver Linton & Esfandiar Maasoumi & Yoon-Jae Whang, 2002. "Consistent Testing for Stochastic Dominance: A Subsampling Approach," STICERD - Econometrics Paper Series 433, Suntory and Toyota International Centres for Economics and Related Disciplines, LSE.
    4. Goncalves, Silvia & White, Halbert, 2004. "Maximum likelihood and the bootstrap for nonlinear dynamic models," Journal of Econometrics, Elsevier, vol. 119(1), pages 199-219, March.
    5. Raffaella Giacomini, 2002. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests: Asymptotic and Bootstrap Methods," Boston College Working Papers in Economics 583, Boston College Department of Economics.
    6. Hansen, Peter Reinhard, 2005. "A Test for Superior Predictive Ability," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 365-380, October.
    7. Granger, C. W. J. & White, Halbert & Kamstra, Mark, 1989. "Interval forecasting : An analysis based upon ARCH-quantile estimators," Journal of Econometrics, Elsevier, vol. 40(1), pages 87-96, January.
    8. Corradi, Valentina & Swanson, Norman R., 2006. "Bootstrap conditional distribution tests in the presence of dynamic misspecification," Journal of Econometrics, Elsevier, vol. 133(2), pages 779-806, August.
    9. Donald W. K. Andrews, 1997. "A Conditional Kolmogorov Test," Econometrica, Econometric Society, vol. 65(5), pages 1097-1128, September.
    10. Yongsung Chang & Joao F. Gomes & Frank Schorfheide, 2002. "Learning-by-Doing as a Propagation Mechanism," American Economic Review, American Economic Association, vol. 92(5), pages 1498-1520, December.
    11. Peter F. Christoffersen & Francis X. Diebold, 2000. "How Relevant is Volatility Forecasting for Financial Risk Management?," The Review of Economics and Statistics, MIT Press, vol. 82(1), pages 12-22, February.
    12. Li, Fuchun & Tkacz, Greg, 2006. "A consistent bootstrap test for conditional density functions with time-series data," Journal of Econometrics, Elsevier, vol. 133(2), pages 863-886, August.
    13. Vuong, Quang H, 1989. "Likelihood Ratio Tests for Model Selection and Non-nested Hypotheses," Econometrica, Econometric Society, vol. 57(2), pages 307-333, March.
    14. Andrews, Donald W K, 1991. "Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation," Econometrica, Econometric Society, vol. 59(3), pages 817-858, May.
    15. Frank Schorfheide, 2000. "Loss function-based evaluation of DSGE models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 15(6), pages 645-670.
    16. Fuchun Li & Greg Tkacz, 2001. "A Consistent Bootstrap Test for Conditional Density Functions with Time-Dependent Data," Staff Working Papers 01-21, Bank of Canada.
    17. Chatfield, Chris, 1993. "Calculating Interval Forecasts," Journal of Business & Economic Statistics, American Statistical Association, vol. 11(2), pages 121-135, April.
    18. Whang, Yoon-Jae, 2001. "Consistent specification testing for conditional moment restrictions," Economics Letters, Elsevier, vol. 71(3), pages 299-306, June.
    19. Amisano, Gianni & Giacomini, Raffaella, 2007. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests," Journal of Business & Economic Statistics, American Statistical Association, vol. 25, pages 177-190, April.
    20. Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-862, November.
    21. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
    22. Clements, Michael P. & Taylor, Nick, 2001. "Bootstrapping prediction intervals for autoregressive models," International Journal of Forecasting, Elsevier, vol. 17(2), pages 247-267.
    23. Francis X. Diebold & Jinyong Hahn & Anthony S. Tay, 1999. "Multivariate Density Forecast Evaluation And Calibration In Financial Risk Management: High-Frequency Returns On Foreign Exchange," The Review of Economics and Statistics, MIT Press, vol. 81(4), pages 661-673, November.
    24. Whang, Yoon-Jae, 2000. "Consistent bootstrap tests of parametric regression functions," Journal of Econometrics, Elsevier, vol. 98(1), pages 27-46, September.
    25. Fernandez-Villaverde, Jesus & Francisco Rubio-Ramirez, Juan, 2004. "Comparing dynamic equilibrium models to data: a Bayesian approach," Journal of Econometrics, Elsevier, vol. 123(1), pages 153-187, November.
    26. White, Halbert, 1982. "Maximum Likelihood Estimation of Misspecified Models," Econometrica, Econometric Society, vol. 50(1), pages 1-25, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Goncalves, Silvia & White, Halbert, 2004. "Maximum likelihood and the bootstrap for nonlinear dynamic models," Journal of Econometrics, Elsevier, vol. 119(1), pages 199-219, March.
    2. Valentina Corradi & Norman Swanson, 2013. "A Survey of Recent Advances in Forecast Accuracy Comparison Testing, with an Extension to Stochastic Dominance," Departmental Working Papers 201309, Rutgers University, Department of Economics.
    3. Mehmet Balcilar & Rangan Gupta & Stephen M. Miller, 2012. "The Out-of-Sample Forecasting Performance of Non-Linear Models of Regional Housing Prices in the US," Working Papers 15-27, Eastern Mediterranean University, Department of Economics.
    4. Shamiri, Ahmed & Shaari, Abu Hassan & Isa, Zaidi, 2007. "Practical Volatility Modeling for Financial Market Risk Management," MPRA Paper 9790, University Library of Munich, Germany, revised 15 May 2008.
    5. Corradi, Valentina & Swanson, Norman R., 2007. "Evaluation of dynamic stochastic general equilibrium models based on distributional comparison of simulated and historical data," Journal of Econometrics, Elsevier, vol. 136(2), pages 699-723, February.
    6. Xiaohong Chen & Sydney C. Ludvigson, 2009. "Land of addicts? an empirical investigation of habit-based asset pricing models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 24(7), pages 1057-1093.
    7. Corradi, Valentina & Swanson, Norman R., 2006. "Predictive density and conditional confidence interval accuracy tests," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 187-228.
    8. Clements, Michael P. & Franses, Philip Hans & Swanson, Norman R., 2004. "Forecasting economic and financial time-series with non-linear models," International Journal of Forecasting, Elsevier, vol. 20(2), pages 169-183.
    9. Corradi, Valentina & Swanson, Norman R., 2011. "Predictive density construction and accuracy testing with multiple possibly misspecified diffusion models," Journal of Econometrics, Elsevier, vol. 161(2), pages 304-324, April.
    10. Norman Swanson & Oleg Korenok, 2006. "How Sticky Is Sticky Enough? A Distributional and Impulse Response Analysis of New Keynesian DSGE Models. Extended Working Paper Version," Departmental Working Papers 200612, Rutgers University, Department of Economics.
    11. Corradi, Valentina & Swanson, Norman R., 2006. "Bootstrap conditional distribution tests in the presence of dynamic misspecification," Journal of Econometrics, Elsevier, vol. 133(2), pages 779-806, August.
    12. Cai, Lili & Swanson, Norman R., 2011. "In- and out-of-sample specification analysis of spot rate models: Further evidence for the period 1982-2008," Journal of Empirical Finance, Elsevier, vol. 18(4), pages 743-764, September.
    13. Chen, Xiaohong & Fan, Yanqin, 2006. "Estimation and model selection of semiparametric copula-based multivariate dynamic models under copula misspecification," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 125-154.
    14. Valentina Corradi & Norman Swanson, 2003. "The Block Bootstrap for Parameter Estimation Error In Recursive Estimation Schemes, With Applications to Predictive Evaluation," Departmental Working Papers 200313, Rutgers University, Department of Economics.
    15. repec:hal:journl:peer-00796745 is not listed on IDEAS
    16. Oleg Korenok & Stanislav Radchenko & Norman R. Swanson, 2010. "International evidence on the efficacy of new-Keynesian models of inflation persistence," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(1), pages 31-54.
    17. Rapach, David E. & Wohar, Mark E., 2006. "The out-of-sample forecasting performance of nonlinear models of real exchange rate behavior," International Journal of Forecasting, Elsevier, vol. 22(2), pages 341-361.
    18. Bhardwaj, Geetesh & Corradi, Valentina & Swanson, Norman R., 2008. "A Simulation-Based Specification Test for Diffusion Processes," Journal of Business & Economic Statistics, American Statistical Association, vol. 26, pages 176-193, April.
    19. Amisano, Gianni & Giacomini, Raffaella, 2007. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests," Journal of Business & Economic Statistics, American Statistical Association, vol. 25, pages 177-190, April.

    More about this item

    Keywords

    block bootstrap; conditional Kolmogorov test; data snooping; misspecified conditional distribution;

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:rut:rutres:200314. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (). General contact details of provider: http://edirc.repec.org/data/derutus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.