IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Log in (now much improved!) to save this paper

A Survey of Recent Advances in Forecast Accuracy Comparison Testing, with an Extension to Stochastic Dominance

Listed author(s):
  • Valentina Corradi

    ()

    (Warwick University)

  • Norman Swanson

    ()

    (Rutgers University)

In recent years, an impressive body or research on predictive accuracy testing and model comparison has been published in the econometrics discipline. Key contributions to this literature include the paper by Diebold and Mariano (DM: 1995) that sets the groundwork for much of the subsequent work in the area, West (1996) who considers a variant of the DM test that allows for parameter estimation error in certain contexts, and White (2000) who develops testing methodology suitable for comparing many models. In this chapter, we begin by reviewing various key testing results in the extant literature, both under vanishing and non-vanishing parameter estimation error, with focus on the construction of valid bootstrap critical values in the case of non-vanishing parameter estimation error, under recursive estimation schemes, drawing on Corradi and Swanson (2007a). We then review recent extensions to the evaluation of multiple confidence intervals and predictive densities, for both the case of a known conditional distribution (Corradi and Swanson 2006a,b) and of an unknown conditional distribution (Corradi and Swanson 2007b). Finally, we introduce a novel approach in which forecast combinations are evaluated via the examination of the quantiles of the expected loss distribution. More precisely, we compare models looking at cumulative distribution functions (CDFs) of prediction errors, for a given loss function, via the principle of stochastic dominance; and we choose the model whose CDF is stochastically dominated, over some given range of interest.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.sas.rutgers.edu/virtual/snde/wp/2013-09.pdf
Download Restriction: no

Paper provided by Rutgers University, Department of Economics in its series Departmental Working Papers with number 201309.

as
in new window

Length: 20 pages
Date of creation: 15 Jul 2013
Handle: RePEc:rut:rutres:201309
Contact details of provider: Postal:
New Jersey Hall - 75 Hamilton Street, New Brunswick, NJ 08901-1248

Phone: (732) 932-7363
Fax: (732) 932-7416
Web page: http://economics.rutgers.edu/

More information through EDIRC

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as
in new window

  1. Jeremy Smith & Kenneth F. Wallis, 2009. "A Simple Explanation of the Forecast Combination Puzzle," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 71(3), pages 331-355, 06.
  2. Allan Timmermann & Halbert White & Ryan Sullivan, 1998. "Data-Snooping, Technical Trading, Rule Performance and the Bootstrap," FMG Discussion Papers dp303, Financial Markets Group.
  3. Goncalves, Silvia & White, Halbert, 2004. "Maximum likelihood and the bootstrap for nonlinear dynamic models," Journal of Econometrics, Elsevier, vol. 119(1), pages 199-219, March.
  4. Graham Elliott & Allan Timmermann, 2005. "Optimal Forecast Combination Under Regime Switching ," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 46(4), pages 1081-1102, November.
  5. Hall, Peter & Horowitz, Joel L, 1996. "Bootstrap Critical Values for Tests Based on Generalized-Method-of-Moments Estimators," Econometrica, Econometric Society, vol. 64(4), pages 891-916, July.
  6. Francis X. Diebold & Lee E. Ohanian & Jeremy Berkowitz, 1995. "Dynamic Equilibrium Economies: A Framework for Comparing Models and Data," NBER Technical Working Papers 0174, National Bureau of Economic Research, Inc.
  7. Gianni Amisano & Raffaella Giacomini, 2005. "Comparing Density Forecsts via Weighted Likelihood Ratio Tests," Working Papers ubs0504, University of Brescia, Department of Economics.
  8. Graham Elliott & Ivana Komunjer & Allan Timmermann, 2005. "Biases In Macroeconomic Forecasts: Irrationality Or Asymmetric Loss?," CAMA Working Papers 2005-14, Centre for Applied Macroeconomic Analysis, Crawford School of Public Policy, The Australian National University.
  9. Yongsung Chang & Joao Gomes & Frank Schorfheide, 2002. "Learning by Doing as a Propagation Mechanism," Macroeconomics 0204002, EconWPA.
  10. Frank Schorfheide, 2000. "Loss function-based evaluation of DSGE models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 15(6), pages 645-670.
  11. Oliver Linton & Esfandiar Maasoumi & Yoon-Jae Whang, 2005. "Consistent Testing for Stochastic Dominance under General Sampling Schemes," Review of Economic Studies, Oxford University Press, vol. 72(3), pages 735-765.
  12. Carlos Capistrán & Allan Timmermann, 2006. "Disagreement and Biases in Inflation Expectations," Working Papers 2006-07, Banco de México.
  13. Diebold, Francis X & Mariano, Roberto S, 1995. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(3), pages 253-263, July.
  14. Giacomini, Raffaella & Komunjer, Ivana, 2002. "Evaluation and Combination of Conditional Quantile Forecasts," University of California at San Diego, Economics Working Paper Series qt4n99t4wz, Department of Economics, UC San Diego.
  15. King, Robert G. & Plosser, Charles I. & Rebelo, Sergio T., 1988. "Production, growth and business cycles : I. The basic neoclassical model," Journal of Monetary Economics, Elsevier, vol. 21(2-3), pages 195-232.
  16. Stephanie Schmitt-Grohe, 1998. "Endogenous business cycles and the dynamics of output, hours, and consumption," Finance and Economics Discussion Series 1998-19, Board of Governors of the Federal Reserve System (U.S.).
  17. Hansen, Peter Reinhard, 2005. "A Test for Superior Predictive Ability," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 365-380, October.
  18. Donald W.K. Andrews, 1999. "Higher-Order Improvements of a Computationally Attractive-Step Bootstrap for Extremum Estimators," Cowles Foundation Discussion Papers 1230R, Cowles Foundation for Research in Economics, Yale University, revised Jan 2001.
  19. Kenneth D. West, 1994. "Asymptotic Inference About Predictive Ability," Macroeconomics 9410002, EconWPA.
  20. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
  21. King, Robert G. & Plosser, Charles I. & Rebelo, Sergio T., 1988. "Production, growth and business cycles : II. New directions," Journal of Monetary Economics, Elsevier, vol. 21(2-3), pages 309-341.
  22. Watson, Mark W, 1993. "Measures of Fit for Calibrated Models," Journal of Political Economy, University of Chicago Press, vol. 101(6), pages 1011-1041, December.
  23. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
  24. Long, John B, Jr & Plosser, Charles I, 1983. "Real Business Cycles," Journal of Political Economy, University of Chicago Press, vol. 91(1), pages 39-69, February.
  25. Valentina Corradi & Norman R. Swanson, 2003. "A Test for Comparing Multiple Misspecified Conditional Distributions," Departmental Working Papers 200314, Rutgers University, Department of Economics.
  26. Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-862, November.
  27. Elliott, Graham & Timmermann, Allan, 2004. "Optimal forecast combinations under general loss functions and forecast error distributions," Journal of Econometrics, Elsevier, vol. 122(1), pages 47-79, September.
  28. Joseph P. Romano & Michael Wolf, 2003. "Stepwise multiple testing as formalized data snooping," Economics Working Papers 712, Department of Economics and Business, Universitat Pompeu Fabra.
  29. Peter F. Christoffersen & Francis X. Diebold, 2000. "How Relevant is Volatility Forecasting for Financial Risk Management?," The Review of Economics and Statistics, MIT Press, vol. 82(1), pages 12-22, February.
  30. West, K.D. & McCracken, M.W., 1997. "Regression-Based Tests of Predictive Ability," Working papers 9710, Wisconsin Madison - Social Systems.
  31. Christoffersen, Peter F & Diebold, Francis X, 1996. "Further Results on Forecasting and Model Selection under Asymmetric Loss," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 11(5), pages 561-571, Sept.-Oct.
  32. Peter R. Hansen & Asger Lunde & James M. Nason, 2010. "The Model Confidence Set," CREATES Research Papers 2010-76, Department of Economics and Business Economics, Aarhus University.
  33. Donald W. K. Andrews & Gustavo Soares, 2010. "Inference for Parameters Defined by Moment Inequalities Using Generalized Moment Selection," Econometrica, Econometric Society, vol. 78(1), pages 119-157, 01.
  34. Bierens, Herman J., 2007. "Econometric analysis of linearized singular dynamic stochastic general equilibrium models," Journal of Econometrics, Elsevier, vol. 136(2), pages 595-627, February.
  35. Kydland, Finn E & Prescott, Edward C, 1982. "Time to Build and Aggregate Fluctuations," Econometrica, Econometric Society, vol. 50(6), pages 1345-1370, November.
  36. Norman Swanson & Valentina Corradi, 2006. "Nonparametric Bootstrap Procedures for Predictive Inference Based on Recursive Estimation Schemes," Departmental Working Papers 200618, Rutgers University, Department of Economics.
  37. Christiano, Lawrence J., 1988. "Why does inventory investment fluctuate so much?," Journal of Monetary Economics, Elsevier, vol. 21(2-3), pages 247-280.
  38. White, Halbert, 1982. "Maximum Likelihood Estimation of Misspecified Models," Econometrica, Econometric Society, vol. 50(1), pages 1-25, January.
  39. Corradi, Valentina & Swanson, Norman R., 2007. "Evaluation of dynamic stochastic general equilibrium models based on distributional comparison of simulated and historical data," Journal of Econometrics, Elsevier, vol. 136(2), pages 699-723, February.
  40. Timothy Cogley & James M. Nason, 1993. "Output dynamics in real business cycle models," Working Papers in Applied Economic Theory 93-10, Federal Reserve Bank of San Francisco.
  41. Atsushi Inoue & Mototsugu Shintani, 2001. "Bootstrapping GMM Estimators for Time Series," Vanderbilt University Department of Economics Working Papers 0129, Vanderbilt University Department of Economics, revised Aug 2003.
  42. Vuong, Quang H, 1989. "Likelihood Ratio Tests for Model Selection and Non-nested Hypotheses," Econometrica, Econometric Society, vol. 57(2), pages 307-333, March.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:rut:rutres:201309. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ()

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.