IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Login to save this paper or follow this series

A Survey of Recent Advances in Forecast Accuracy Comparison Testing, with an Extension to Stochastic Dominance

  • Valentina Corradi

    ()

    (Warwick University)

  • Norman Swanson

    ()

    (Rutgers University)

In recent years, an impressive body or research on predictive accuracy testing and model comparison has been published in the econometrics discipline. Key contributions to this literature include the paper by Diebold and Mariano (DM: 1995) that sets the groundwork for much of the subsequent work in the area, West (1996) who considers a variant of the DM test that allows for parameter estimation error in certain contexts, and White (2000) who develops testing methodology suitable for comparing many models. In this chapter, we begin by reviewing various key testing results in the extant literature, both under vanishing and non-vanishing parameter estimation error, with focus on the construction of valid bootstrap critical values in the case of non-vanishing parameter estimation error, under recursive estimation schemes, drawing on Corradi and Swanson (2007a). We then review recent extensions to the evaluation of multiple confidence intervals and predictive densities, for both the case of a known conditional distribution (Corradi and Swanson 2006a,b) and of an unknown conditional distribution (Corradi and Swanson 2007b). Finally, we introduce a novel approach in which forecast combinations are evaluated via the examination of the quantiles of the expected loss distribution. More precisely, we compare models looking at cumulative distribution functions (CDFs) of prediction errors, for a given loss function, via the principle of stochastic dominance; and we choose the model whose CDF is stochastically dominated, over some given range of interest.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: ftp://snde.rutgers.edu/Rutgers/wp/2013-09.pdf
Download Restriction: no

Paper provided by Rutgers University, Department of Economics in its series Departmental Working Papers with number 201309.

as
in new window

Length: 20 pages
Date of creation: 15 Jul 2013
Date of revision:
Handle: RePEc:rut:rutres:201309
Contact details of provider: Postal: New Jersey Hall - 75 Hamilton Street, New Brunswick, NJ 08901-1248
Phone: (732) 932-7482
Fax: (732) 932-7416
Web page: http://snde.rutgers.edu/Rutgers/wp/rutgers-wplist.html

More information through EDIRC

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Goncalves, Silvia & White, Halbert, 2004. "Maximum likelihood and the bootstrap for nonlinear dynamic models," Journal of Econometrics, Elsevier, vol. 119(1), pages 199-219, March.
  2. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
  3. Donald W.K. Andrews & Gustavo Soares, 2007. "Inference for Parameters Defined by Moment Inequalities Using Generalized Moment Selection," Cowles Foundation Discussion Papers 1631, Cowles Foundation for Research in Economics, Yale University.
  4. Amisano, Gianni & Giacomini, Raffaella, 2007. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests," Journal of Business & Economic Statistics, American Statistical Association, vol. 25, pages 177-190, April.
  5. Yongsung Chang & Joao F. Gomes & Frank Schorfheide, 2002. "Learning-by-Doing as a Propagation Mechanism," American Economic Review, American Economic Association, vol. 92(5), pages 1498-1520, December.
  6. Long, John B, Jr & Plosser, Charles I, 1983. "Real Business Cycles," Journal of Political Economy, University of Chicago Press, vol. 91(1), pages 39-69, February.
  7. Schmitt-Grohé, Stephanie, 1999. "Endogenous Business Cycles and the Dynamics of Output, Hours, and Consumption," CEPR Discussion Papers 2315, C.E.P.R. Discussion Papers.
  8. Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-62, November.
  9. Kydland, Finn E & Prescott, Edward C, 1982. "Time to Build and Aggregate Fluctuations," Econometrica, Econometric Society, vol. 50(6), pages 1345-70, November.
  10. Carlos Capistrán & Allan Timmermann, 2006. "Disagreement and Biases in Inflation Expectations," Working Papers 2006-07, Banco de México.
  11. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-44, January.
  12. Francis X. Diebold & Lee E. Ohanian & Jeremy Berkowitz, 1995. "Dynamic Equilibrium Economies: A Framework for Comparing Models and Data," NBER Technical Working Papers 0174, National Bureau of Economic Research, Inc.
  13. Vuong, Quang H, 1989. "Likelihood Ratio Tests for Model Selection and Non-nested Hypotheses," Econometrica, Econometric Society, vol. 57(2), pages 307-33, March.
  14. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
  15. Bierens, Herman J., 2007. "Econometric analysis of linearized singular dynamic stochastic general equilibrium models," Journal of Econometrics, Elsevier, vol. 136(2), pages 595-627, February.
  16. Peter F. Christoffersen & Francis X. Diebold, 2000. "How Relevant is Volatility Forecasting for Financial Risk Management?," The Review of Economics and Statistics, MIT Press, vol. 82(1), pages 12-22, February.
  17. West, Kenneth D & McCracken, Michael W, 1998. "Regression-Based Tests of Predictive Ability," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 817-40, November.
  18. Inoue, Atsushi & Shintani, Mototsugu, 2006. "Bootstrapping GMM estimators for time series," Journal of Econometrics, Elsevier, vol. 133(2), pages 531-555, August.
  19. Norman Swanson & Valentina Corradi, 2006. "Nonparametric Bootstrap Procedures for Predictive Inference Based on Recursive Estimation Schemes," Departmental Working Papers 200618, Rutgers University, Department of Economics.
  20. Jeremy Smith & Kenneth F. Wallis, 2009. "A Simple Explanation of the Forecast Combination Puzzle," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 71(3), pages 331-355, 06.
  21. Elliott, Graham & Timmermann, Allan G, 2004. "Optimal Forecast Combination Under Regime Switching," CEPR Discussion Papers 4649, C.E.P.R. Discussion Papers.
  22. King, Robert G. & Plosser, Charles I. & Rebelo, Sergio T., 1988. "Production, growth and business cycles : II. New directions," Journal of Monetary Economics, Elsevier, vol. 21(2-3), pages 309-341.
  23. Raffaella Giacomini & Ivana Komunjer, 2003. "Evaluation and Combination of Conditional Quantile Forecasts," Boston College Working Papers in Economics 571, Boston College Department of Economics.
  24. Valentina Corradi & Norman R. Swanson, 2003. "Evaluation of Dynamic Stochastic General Equilibrium Models Based on Distributional Comparison of Simulated and Historical Data," Departmental Working Papers 200320, Rutgers University, Department of Economics.
  25. Allan Timmermann & Halbert White & Ryan Sullivan, 1998. "Data-Snooping, Technical Trading, Rule Performance and the Bootstrap," FMG Discussion Papers dp303, Financial Markets Group.
  26. Cogley, Timothy & Nason, James M, 1995. "Output Dynamics in Real-Business-Cycle Models," American Economic Review, American Economic Association, vol. 85(3), pages 492-511, June.
  27. Allan Timmermann & Graham Elliott & Ivana Komunjer, 2004. "Biases in Macroeconomic Forecasts: Irrationality or Asymmetric Loss?," Econometric Society 2004 North American Summer Meetings 601, Econometric Society.
  28. Donald W.K. Andrews, 1999. "Higher-Order Improvements of a Computationally Attractive-Step Bootstrap for Extremum Estimators," Cowles Foundation Discussion Papers 1230R, Cowles Foundation for Research in Economics, Yale University, revised Jan 2001.
  29. Joseph P. Romano & Michael Wolf, 2005. "Stepwise Multiple Testing as Formalized Data Snooping," Econometrica, Econometric Society, vol. 73(4), pages 1237-1282, 07.
  30. Oliver Linton & Esfandiar Maasoumi & Yoon-Jae Whang, 2005. "Consistent Testing for Stochastic Dominance under General Sampling Schemes," Review of Economic Studies, Oxford University Press, vol. 72(3), pages 735-765.
  31. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-84, September.
  32. Christoffersen, Peter F & Diebold, Francis X, 1996. "Further Results on Forecasting and Model Selection under Asymmetric Loss," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 11(5), pages 561-71, Sept.-Oct.
  33. Peter R. Hansen & Asger Lunde & James M. Nason, 2011. "The Model Confidence Set," Econometrica, Econometric Society, vol. 79(2), pages 453-497, 03.
  34. Elliott, Graham & Timmermann, Allan, 2004. "Optimal forecast combinations under general loss functions and forecast error distributions," Journal of Econometrics, Elsevier, vol. 122(1), pages 47-79, September.
  35. Valentina Corradi & Norman R. Swanson, 2003. "A Test for Comparing Multiple Misspecified Conditional Distributions," Departmental Working Papers 200314, Rutgers University, Department of Economics.
  36. Hall, Peter & Horowitz, Joel L, 1996. "Bootstrap Critical Values for Tests Based on Generalized-Method-of-Moments Estimators," Econometrica, Econometric Society, vol. 64(4), pages 891-916, July.
  37. White, Halbert, 1982. "Maximum Likelihood Estimation of Misspecified Models," Econometrica, Econometric Society, vol. 50(1), pages 1-25, January.
  38. Christiano, Lawrence J., 1988. "Why does inventory investment fluctuate so much?," Journal of Monetary Economics, Elsevier, vol. 21(2-3), pages 247-280.
  39. Mark W. Watson, 1991. "Measures of fit for calibrated models," Working Paper Series, Macroeconomic Issues 91-9, Federal Reserve Bank of Chicago.
  40. Frank Schorfheide, 2000. "Loss function-based evaluation of DSGE models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 15(6), pages 645-670.
  41. King, Robert G. & Plosser, Charles I. & Rebelo, Sergio T., 1988. "Production, growth and business cycles : I. The basic neoclassical model," Journal of Monetary Economics, Elsevier, vol. 21(2-3), pages 195-232.
  42. Hansen, Peter Reinhard, 2005. "A Test for Superior Predictive Ability," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 365-380, October.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:rut:rutres:201309. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ()

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.