IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Login to save this paper or follow this series

A Survey of Recent Advances in Forecast Accuracy Comparison Testing, with an Extension to Stochastic Dominance

  • Valentina Corradi

    ()

    (Warwick University)

  • Norman Swanson

    ()

    (Rutgers University)

In recent years, an impressive body or research on predictive accuracy testing and model comparison has been published in the econometrics discipline. Key contributions to this literature include the paper by Diebold and Mariano (DM: 1995) that sets the groundwork for much of the subsequent work in the area, West (1996) who considers a variant of the DM test that allows for parameter estimation error in certain contexts, and White (2000) who develops testing methodology suitable for comparing many models. In this chapter, we begin by reviewing various key testing results in the extant literature, both under vanishing and non-vanishing parameter estimation error, with focus on the construction of valid bootstrap critical values in the case of non-vanishing parameter estimation error, under recursive estimation schemes, drawing on Corradi and Swanson (2007a). We then review recent extensions to the evaluation of multiple confidence intervals and predictive densities, for both the case of a known conditional distribution (Corradi and Swanson 2006a,b) and of an unknown conditional distribution (Corradi and Swanson 2007b). Finally, we introduce a novel approach in which forecast combinations are evaluated via the examination of the quantiles of the expected loss distribution. More precisely, we compare models looking at cumulative distribution functions (CDFs) of prediction errors, for a given loss function, via the principle of stochastic dominance; and we choose the model whose CDF is stochastically dominated, over some given range of interest.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: ftp://snde.rutgers.edu/Rutgers/wp/2013-09.pdf
Download Restriction: no

Paper provided by Rutgers University, Department of Economics in its series Departmental Working Papers with number 201309.

as
in new window

Length: 20 pages
Date of creation: 15 Jul 2013
Date of revision:
Handle: RePEc:rut:rutres:201309
Contact details of provider: Postal: New Jersey Hall - 75 Hamilton Street, New Brunswick, NJ 08901-1248
Phone: (732) 932-7482
Fax: (732) 932-7416
Web page: http://snde.rutgers.edu/Rutgers/wp/rutgers-wplist.html
More information through EDIRC

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-44, January.
  2. Raffaella Giacomini & Ivana Komunjer, 2003. "Evaluation and Combination of Conditional Quantile Forecasts," Boston College Working Papers in Economics 571, Boston College Department of Economics.
  3. Carlos Capistrán & Allan Timmermann, 2009. "Disagreement and Biases in Inflation Expectations," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 41(2-3), pages 365-396, 03.
  4. Christoffersen, Peter F & Diebold, Francis X, 1996. "Further Results on Forecasting and Model Selection under Asymmetric Loss," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 11(5), pages 561-71, Sept.-Oct.
  5. Peter F. Christoffersen & Francis X. Diebold, 2000. "How Relevant is Volatility Forecasting for Financial Risk Management?," The Review of Economics and Statistics, MIT Press, vol. 82(1), pages 12-22, February.
  6. Donald W.K. Andrews & Gustavo Soares, 2007. "Inference for Parameters Defined by Moment Inequalities Using Generalized Moment Selection," Cowles Foundation Discussion Papers 1631, Cowles Foundation for Research in Economics, Yale University.
  7. Christiano, Lawrence J., 1988. "Why does inventory investment fluctuate so much?," Journal of Monetary Economics, Elsevier, vol. 21(2-3), pages 247-280.
  8. Francis X. Diebold & Lee E. Ohanian & Jeremy Berkowitz, 1995. "Dynamic Equilibrium Economies: A Framework for Comparing Models and Data," NBER Technical Working Papers 0174, National Bureau of Economic Research, Inc.
  9. Graham Elliott & Allan Timmermann, 2005. "Optimal Forecast Combination Under Regime Switching ," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 46(4), pages 1081-1102, November.
  10. Timothy Cogley & James M. Nason, 1993. "Output dynamics in real business cycle models," Working Papers in Applied Economic Theory 93-10, Federal Reserve Bank of San Francisco.
  11. West, Kenneth D & McCracken, Michael W, 1998. "Regression-Based Tests of Predictive Ability," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 817-40, November.
  12. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-84, September.
  13. Todd E. Clark & Michael McCracken, 1999. "Tests of Equal Forecast Accuracy and Encompassing for Nested Models," Computing in Economics and Finance 1999 1241, Society for Computational Economics.
  14. Jeremy Smith & Kenneth F. Wallis, 2009. "A Simple Explanation of the Forecast Combination Puzzle," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 71(3), pages 331-355, 06.
  15. Graham Elliott & Ivana Komunjer & Allan Timmermann, 2008. "Biases in Macroeconomic Forecasts: Irrationality or Asymmetric Loss?," Journal of the European Economic Association, MIT Press, vol. 6(1), pages 122-157, 03.
  16. White, Halbert, 1982. "Maximum Likelihood Estimation of Misspecified Models," Econometrica, Econometric Society, vol. 50(1), pages 1-25, January.
  17. Allan Timmermann & Halbert White & Ryan Sullivan, 1998. "Data-Snooping, Technical Trading, Rule Performance and the Bootstrap," FMG Discussion Papers dp303, Financial Markets Group.
  18. Linton, Oliver & Maasoumi, Esfandiar & Whang, Yoon-Jae, 2003. "Consistent Testing for Stochastic Dominance under General Sampling Schemes," SFB 373 Discussion Papers 2003,31, Humboldt University of Berlin, Interdisciplinary Research Project 373: Quantification and Simulation of Economic Processes.
  19. Mark W. Watson, 1991. "Measures of Fit for Calibrated Models," NBER Technical Working Papers 0102, National Bureau of Economic Research, Inc.
  20. Amisano, Gianni & Giacomini, Raffaella, 2007. "Comparing Density Forecasts via Weighted Likelihood Ratio Tests," Journal of Business & Economic Statistics, American Statistical Association, vol. 25, pages 177-190, April.
  21. Vuong, Quang H, 1989. "Likelihood Ratio Tests for Model Selection and Non-nested Hypotheses," Econometrica, Econometric Society, vol. 57(2), pages 307-33, March.
  22. Peter R. Hansen & Asger Lunde & James M. Nason, 2010. "The Model Confidence Set," CREATES Research Papers 2010-76, School of Economics and Management, University of Aarhus.
  23. Norman Swanson & Valentina Corradi, 2006. "Nonparametric Bootstrap Procedures for Predictive Inference Based on Recursive Estimation Schemes," Departmental Working Papers 200618, Rutgers University, Department of Economics.
  24. Chang, Yongsung & Gomes, Joao F & Schorfheide, Frank, 2002. "Learning by Doing as a Propagation Mechanism," CEPR Discussion Papers 3599, C.E.P.R. Discussion Papers.
  25. Goncalves, Silvia & White, Halbert, 2000. "Maximum Likelihood and the Bootstrap for Nonlinear Dynamic Models," University of California at San Diego, Economics Working Paper Series qt1bj657ff, Department of Economics, UC San Diego.
  26. Hansen, Peter Reinhard, 2005. "A Test for Superior Predictive Ability," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 365-380, October.
  27. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
  28. Valentina Corradi & Norman R. Swanson, 2003. "A Test for Comparing Multiple Misspecified Conditional Distributions," Departmental Working Papers 200314, Rutgers University, Department of Economics.
  29. Donald W. K. Andrews, 2002. "Higher-Order Improvements of a Computationally Attractive "k"-Step Bootstrap for Extremum Estimators," Econometrica, Econometric Society, vol. 70(1), pages 119-162, January.
  30. King, Robert G. & Plosser, Charles I. & Rebelo, Sergio T., 1988. "Production, growth and business cycles : I. The basic neoclassical model," Journal of Monetary Economics, Elsevier, vol. 21(2-3), pages 195-232.
  31. Bierens, Herman J., 2007. "Econometric analysis of linearized singular dynamic stochastic general equilibrium models," Journal of Econometrics, Elsevier, vol. 136(2), pages 595-627, February.
  32. Inoue, Atsushi & Shintani, Mototsugu, 2006. "Bootstrapping GMM estimators for time series," Journal of Econometrics, Elsevier, vol. 133(2), pages 531-555, August.
  33. King, Robert G. & Plosser, Charles I. & Rebelo, Sergio T., 1988. "Production, growth and business cycles : II. New directions," Journal of Monetary Economics, Elsevier, vol. 21(2-3), pages 309-341.
  34. Valentina Corradi & Norman R. Swanson, 2003. "Evaluation of Dynamic Stochastic General Equilibrium Models Based on Distributional Comparison of Simulated and Historical Data," Departmental Working Papers 200320, Rutgers University, Department of Economics.
  35. Kydland, Finn E & Prescott, Edward C, 1982. "Time to Build and Aggregate Fluctuations," Econometrica, Econometric Society, vol. 50(6), pages 1345-70, November.
  36. Joseph P. Romano & Michael Wolf, 2005. "Stepwise Multiple Testing as Formalized Data Snooping," Econometrica, Econometric Society, vol. 73(4), pages 1237-1282, 07.
  37. Stephanie Schmitt-Grohe, 1999. "Endogenous business cycles and the dynamics of output, hours, and consumption," Departmental Working Papers 199915, Rutgers University, Department of Economics.
  38. Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-62, November.
  39. Elliott, Graham & Timmermann, Allan, 2004. "Optimal forecast combinations under general loss functions and forecast error distributions," Journal of Econometrics, Elsevier, vol. 122(1), pages 47-79, September.
  40. Long, John B, Jr & Plosser, Charles I, 1983. "Real Business Cycles," Journal of Political Economy, University of Chicago Press, vol. 91(1), pages 39-69, February.
  41. Frank Schorfheide, 2000. "Loss function-based evaluation of DSGE models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 15(6), pages 645-670.
  42. Hall, Peter & Horowitz, Joel L, 1996. "Bootstrap Critical Values for Tests Based on Generalized-Method-of-Moments Estimators," Econometrica, Econometric Society, vol. 64(4), pages 891-916, July.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:rut:rutres:201309. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ()

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.