IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Login to save this paper or follow this series

A Survey of Recent Advances in Forecast Accuracy Comparison Testing, with an Extension to Stochastic Dominance

  • Valentina Corradi

    ()

    (Warwick University)

  • Norman Swanson

    ()

    (Rutgers University)

In recent years, an impressive body or research on predictive accuracy testing and model comparison has been published in the econometrics discipline. Key contributions to this literature include the paper by Diebold and Mariano (DM: 1995) that sets the groundwork for much of the subsequent work in the area, West (1996) who considers a variant of the DM test that allows for parameter estimation error in certain contexts, and White (2000) who develops testing methodology suitable for comparing many models. In this chapter, we begin by reviewing various key testing results in the extant literature, both under vanishing and non-vanishing parameter estimation error, with focus on the construction of valid bootstrap critical values in the case of non-vanishing parameter estimation error, under recursive estimation schemes, drawing on Corradi and Swanson (2007a). We then review recent extensions to the evaluation of multiple confidence intervals and predictive densities, for both the case of a known conditional distribution (Corradi and Swanson 2006a,b) and of an unknown conditional distribution (Corradi and Swanson 2007b). Finally, we introduce a novel approach in which forecast combinations are evaluated via the examination of the quantiles of the expected loss distribution. More precisely, we compare models looking at cumulative distribution functions (CDFs) of prediction errors, for a given loss function, via the principle of stochastic dominance; and we choose the model whose CDF is stochastically dominated, over some given range of interest.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: ftp://snde.rutgers.edu/Rutgers/wp/2013-09.pdf
Download Restriction: no

Paper provided by Rutgers University, Department of Economics in its series Departmental Working Papers with number 201309.

as
in new window

Length: 20 pages
Date of creation: 15 Jul 2013
Date of revision:
Handle: RePEc:rut:rutres:201309
Contact details of provider: Postal: New Jersey Hall - 75 Hamilton Street, New Brunswick, NJ 08901-1248
Phone: (732) 932-7363
Fax: (732) 932-7416
Web page: http://snde.rutgers.edu/Rutgers/wp/rutgers-wplist.html

More information through EDIRC

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Norman Swanson & Valentina Corradi, 2006. "Nonparametric Bootstrap Procedures for Predictive Inference Based on Recursive Estimation Schemes," Departmental Working Papers 200618, Rutgers University, Department of Economics.
  2. Corradi, Valentina & Swanson, Norman R., 2007. "Evaluation of dynamic stochastic general equilibrium models based on distributional comparison of simulated and historical data," Journal of Econometrics, Elsevier, vol. 136(2), pages 699-723, February.
  3. Kenneth D. West & Michael W. McCracken, 1998. "Regression-Based Tests of Predictive Ability," NBER Technical Working Papers 0226, National Bureau of Economic Research, Inc.
  4. Elliott, Graham & Timmermann, Allan, 2002. "Optimal Forecast Combination Under General Loss Functions and Forecast Error Distributions," University of California at San Diego, Economics Working Paper Series qt15r9t2q2, Department of Economics, UC San Diego.
  5. Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-62, November.
  6. repec:att:wimass:9417 is not listed on IDEAS
  7. Stephanie Schmitt-Grohe, 2000. "Endogenous Business Cycles and the Dynamics of Output, Hours, and Consumption," American Economic Review, American Economic Association, vol. 90(5), pages 1136-1159, December.
  8. Todd E. Clark & Michael W. McCracken, 2000. "Tests of Equal Forecast Accuracy and Encompassing for Nested Models," Econometric Society World Congress 2000 Contributed Papers 0319, Econometric Society.
  9. Goncalves, Silvia & White, Halbert, 2000. "Maximum Likelihood and the Bootstrap for Nonlinear Dynamic Models," University of California at San Diego, Economics Working Paper Series qt1bj657ff, Department of Economics, UC San Diego.
  10. Linton, Oliver & Maasoumi, Esfandiar & Whang, Yoon-Jae, 2003. "Consistent Testing for Stochastic Dominance under General Sampling Schemes," SFB 373 Discussion Papers 2003,31, Humboldt University of Berlin, Interdisciplinary Research Project 373: Quantification and Simulation of Economic Processes.
  11. Inoue, Atsushi & Shintani, Mototsugu, 2006. "Bootstrapping GMM estimators for time series," Journal of Econometrics, Elsevier, vol. 133(2), pages 531-555, August.
  12. Carlos Capistrán & Allan Timmermann, 2008. "Disagreement and Biases in Inflation Expectations," CREATES Research Papers 2008-56, School of Economics and Management, University of Aarhus.
  13. Elliott, Graham & Timmermann, Allan G, 2004. "Optimal Forecast Combination Under Regime Switching," CEPR Discussion Papers 4649, C.E.P.R. Discussion Papers.
  14. White, Halbert, 1982. "Maximum Likelihood Estimation of Misspecified Models," Econometrica, Econometric Society, vol. 50(1), pages 1-25, January.
  15. Francis X. Diebold & Robert S. Mariano, 1994. "Comparing Predictive Accuracy," NBER Technical Working Papers 0169, National Bureau of Economic Research, Inc.
  16. Graham Elliott & Ivana Komunjer & Allan Timmermann, 2005. "Biases In Macroeconomic Forecasts: Irrationality Or Asymmetric Loss?," CAMA Working Papers 2005-14, Centre for Applied Macroeconomic Analysis, Crawford School of Public Policy, The Australian National University.
  17. Francis X. Diebold & Lee E. Ohanian & Jeremy Berkowitz, 1995. "Dynamic Equilibrium Economies: A Framework for Comparing Models and Data," NBER Technical Working Papers 0174, National Bureau of Economic Research, Inc.
  18. Mark W. Watson, 1991. "Measures of fit for calibrated models," Working Paper Series, Macroeconomic Issues 91-9, Federal Reserve Bank of Chicago.
  19. Peter F. Christoffersen & Francis X. Diebold, 1998. "How Relevant is Volatility Forecasting for Financial Risk Management?," New York University, Leonard N. Stern School Finance Department Working Paper Seires 98-080, New York University, Leonard N. Stern School of Business-.
  20. Frank Schorfheide, 2000. "Loss function-based evaluation of DSGE models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 15(6), pages 645-670.
  21. Allan Timmermann & Halbert White & Ryan Sullivan, 1998. "Data-Snooping, Technical Trading, Rule Performance and the Bootstrap," FMG Discussion Papers dp303, Financial Markets Group.
  22. Chang, Yongsung & Gomes, Joao F & Schorfheide, Frank, 2002. "Learning by Doing as a Propagation Mechanism," CEPR Discussion Papers 3599, C.E.P.R. Discussion Papers.
  23. Kenneth D. West, 1994. "Asymptotic Inference About Predictive Ability," Macroeconomics 9410002, EconWPA.
  24. Hansen, Peter Reinhard, 2005. "A Test for Superior Predictive Ability," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 365-380, October.
  25. Gianni Amisano & Raffaella Giacomini, 2005. "Comparing Density Forecsts via Weighted Likelihood Ratio Tests," Working Papers ubs0504, University of Brescia, Department of Economics.
  26. Joseph P. Romano & Michael Wolf, 2003. "Stepwise multiple testing as formalized data snooping," Economics Working Papers 712, Department of Economics and Business, Universitat Pompeu Fabra.
  27. Christoffersen & Diebold, . "Further Results on Forecasting and Model Selection Under Asymmetric Loss," Home Pages _059, University of Pennsylvania.
  28. Donald W.K. Andrews, 1999. "Higher-Order Improvements of a Computationally Attractive-Step Bootstrap for Extremum Estimators," Cowles Foundation Discussion Papers 1230R, Cowles Foundation for Research in Economics, Yale University, revised Jan 2001.
  29. Timothy Cogley & James M. Nason, 1993. "Output dynamics in real business cycle models," Working Papers in Applied Economic Theory 93-10, Federal Reserve Bank of San Francisco.
  30. Giacomini, Raffaella & Komunjer, Ivana, 2002. "Evaluation and Combination of Conditional Quantile Forecasts," University of California at San Diego, Economics Working Paper Series qt4n99t4wz, Department of Economics, UC San Diego.
  31. Valentina Corradi & Norman R. Swanson, 2003. "A Test for Comparing Multiple Misspecified Conditional Distributions," Departmental Working Papers 200314, Rutgers University, Department of Economics.
  32. Christiano, Lawrence J., 1988. "Why does inventory investment fluctuate so much?," Journal of Monetary Economics, Elsevier, vol. 21(2-3), pages 247-280.
  33. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
  34. Vuong, Quang H, 1989. "Likelihood Ratio Tests for Model Selection and Non-nested Hypotheses," Econometrica, Econometric Society, vol. 57(2), pages 307-33, March.
  35. Long, John B, Jr & Plosser, Charles I, 1983. "Real Business Cycles," Journal of Political Economy, University of Chicago Press, vol. 91(1), pages 39-69, February.
  36. Peter R. Hansen & Asger Lunde & James M. Nason, 2010. "The Model Confidence Set," CREATES Research Papers 2010-76, School of Economics and Management, University of Aarhus.
  37. Hall, Peter & Horowitz, Joel L, 1996. "Bootstrap Critical Values for Tests Based on Generalized-Method-of-Moments Estimators," Econometrica, Econometric Society, vol. 64(4), pages 891-916, July.
  38. Jeremy Smith & Kenneth F. Wallis, 2009. "A Simple Explanation of the Forecast Combination Puzzle," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 71(3), pages 331-355, 06.
  39. Bierens, Herman J., 2007. "Econometric analysis of linearized singular dynamic stochastic general equilibrium models," Journal of Econometrics, Elsevier, vol. 136(2), pages 595-627, February.
  40. Kydland, Finn E & Prescott, Edward C, 1982. "Time to Build and Aggregate Fluctuations," Econometrica, Econometric Society, vol. 50(6), pages 1345-70, November.
  41. King, Robert G. & Plosser, Charles I. & Rebelo, Sergio T., 1988. "Production, growth and business cycles : II. New directions," Journal of Monetary Economics, Elsevier, vol. 21(2-3), pages 309-341.
  42. Donald W.K. Andrews & Gustavo Soares, 2007. "Inference for Parameters Defined by Moment Inequalities Using Generalized Moment Selection," Cowles Foundation Discussion Papers 1631, Cowles Foundation for Research in Economics, Yale University.
  43. King, Robert G. & Plosser, Charles I. & Rebelo, Sergio T., 1988. "Production, growth and business cycles : I. The basic neoclassical model," Journal of Monetary Economics, Elsevier, vol. 21(2-3), pages 195-232.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:rut:rutres:201309. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ()

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.