IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Login to save this paper or follow this series

Advances in forecast evaluation

  • Todd E. Clark
  • Michael W. McCracken

This paper surveys recent developments in the evaluation of point forecasts. Taking West's (2006) survey as a starting point, we briefly cover the state of the literature as of the time of West's writing. We then focus on recent developments, including advancements in the evaluation of forecasts at the population level (based on true, unknown model coefficients), the evaluation of forecasts in the finite sample (based on estimated model coefficients), and the evaluation of conditional versus unconditional forecasts. We present original results in a few subject areas: the optimization of power in determining the split of a sample into in-sample and out-of-sample portions; whether the accuracy of inference in evaluation of multi-step forecasts can be improved with judicious choice of HAC estimator (it can); and the extension of West's (1996) theory results for population-level, unconditional forecast evaluation to the case of conditional forecast evaluation.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://research.stlouisfed.org/wp/2011/2011-025.pdf
Download Restriction: no

Paper provided by Federal Reserve Bank of St. Louis in its series Working Papers with number 2011-025.

as
in new window

Length:
Date of creation: 2011
Date of revision:
Handle: RePEc:fip:fedlwp:2011-025
Contact details of provider: Postal: P.O. Box 442, St. Louis, MO 63166
Fax: (314)444-8753
Web page: http://www.stlouisfed.org/

More information through EDIRC

Order Information: Email:


References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Aruoba, Boragan, 2005. "Data Revisions Are Not Well-Behaved," CEPR Discussion Papers 5271, C.E.P.R. Discussion Papers.
  2. Clark, Todd E. & West, Kenneth D., 2007. "Approximately normal tests for equal predictive accuracy in nested models," Journal of Econometrics, Elsevier, vol. 138(1), pages 291-311, May.
  3. Clements, Michael P. & Galvão, Ana Beatriz, 2010. "Real-time Forecasting of Inflation and Output Growth in the Presence of Data Revisions," The Warwick Economics Research Paper Series (TWERPS) 953, University of Warwick, Department of Economics.
  4. West, Kenneth D., 1997. "Another heteroskedasticity- and autocorrelation-consistent covariance matrix estimator," Journal of Econometrics, Elsevier, vol. 76(1-2), pages 171-191.
  5. Ray C. Fair & Robert J. Shiller, 1988. "The Informational Content of Ex Ante Forecasts," Cowles Foundation Discussion Papers 857, Cowles Foundation for Research in Economics, Yale University.
  6. Yu-Chin Chen & Kenneth Rogoff & Barbara Rossi, 2008. "Can Exchange Rates Forecast Commodity Prices?," NBER Working Papers 13901, National Bureau of Economic Research, Inc.
  7. Evan F. Koenig & Sheila Dolmas & Jeremy Piger, 2000. "The use and abuse of "real-time" data in economic forecasting," International Finance Discussion Papers 684, Board of Governors of the Federal Reserve System (U.S.).
  8. Athanasios Orphanides & Simon van Norden, 2003. "The Reliability of Inflation Forecasts Based on Output Gap Estimates in Real Time," CIRANO Working Papers 2003s-01, CIRANO.
  9. McCracken, Michael W., 2004. "Parameter estimation and tests of equal forecast accuracy between non-nested models," International Journal of Forecasting, Elsevier, vol. 20(3), pages 503-514.
  10. Giacomini, Raffaella & Rossi, Barbara, 2006. "Detecting and predicting forecast breakdowns," Working Paper Series 0638, European Central Bank.
  11. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-44, January.
  12. Mc Cracken, Michael W., 2000. "Robust out-of-sample inference," Journal of Econometrics, Elsevier, vol. 99(2), pages 195-223, December.
  13. Harvey, David & Leybourne, Stephen & Newbold, Paul, 1997. "Testing the equality of prediction mean squared errors," International Journal of Forecasting, Elsevier, vol. 13(2), pages 281-291, June.
  14. Todd E. Clark & Michael W. McCracken, 2009. "Nested forecast model comparisons: a new approach to testing equal accuracy," Working Papers 2009-050, Federal Reserve Bank of St. Louis.
  15. Fabio Busetti & Juri Marcucci & Giovanni Veronese, 2009. "Comparing forecast accuracy: A Monte Carlo investigation," Temi di discussione (Economic working papers) 723, Bank of Italy, Economic Research and International Relations Area.
  16. Barbara Rossi & Atsushi Inoue, 2011. "Out-of-Sample Forecast Tests Robust to Window Size Choice," Working Papers 11-04, Duke University, Department of Economics.
  17. Todd Clark & Michael McCracken, 2005. "Evaluating Direct Multistep Forecasts," Econometric Reviews, Taylor & Francis Journals, vol. 24(4), pages 369-404.
  18. Todd E. Clark & Michael W. McCracken, 2000. "Tests of Equal Forecast Accuracy and Encompassing for Nested Models," Econometric Society World Congress 2000 Contributed Papers 0319, Econometric Society.
  19. Todd E. Clark & Michael W. McCracken, 2008. "Tests of equal predictive ability with real-time data," Working Papers 2008-029, Federal Reserve Bank of St. Louis.
  20. McCracken, Michael W., 2007. "Asymptotics for out of sample tests of Granger causality," Journal of Econometrics, Elsevier, vol. 140(2), pages 719-752, October.
  21. Amit Goyal & Ivo Welch, 2004. "A Comprehensive Look at the Empirical Performance of Equity Premium Prediction," Yale School of Management Working Papers amz2412, Yale School of Management, revised 01 Jan 2006.
  22. Todd E. Clark & Michael W. McCracken, 2010. "Testing for unconditional predictive ability," Working Papers 2010-031, Federal Reserve Bank of St. Louis.
  23. Clark, Todd E. & West, Kenneth D., 2006. "Using out-of-sample mean squared prediction errors to test the martingale difference hypothesis," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 155-186.
  24. Kilian, Lutz, 1999. "Exchange Rates and Monetary Fundamentals: What Do We Learn from Long-Horizon Regressions?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 14(5), pages 491-510, Sept.-Oct.
  25. Davidson, James & de Jong, Robert M., 2000. "The Functional Central Limit Theorem And Weak Convergence To Stochastic Integrals Ii," Econometric Theory, Cambridge University Press, vol. 16(05), pages 643-666, October.
  26. Ashley, R & Granger, C W J & Schmalensee, R, 1980. "Advertising and Aggregate Consumption: An Analysis of Causality," Econometrica, Econometric Society, vol. 48(5), pages 1149-67, July.
  27. Stanislav Anatolyev, 2007. "Inference about predictive ability when there are many predictors," Working Papers w0096, Center for Economic and Financial Research (CEFIR).
  28. Hansen, Bruce E., 1992. "Convergence to Stochastic Integrals for Dependent Heterogeneous Processes," Econometric Theory, Cambridge University Press, vol. 8(04), pages 489-500, December.
  29. Clark, Todd E. & McCracken, Michael W., 2005. "The power of tests of predictive ability in the presence of structural breaks," Journal of Econometrics, Elsevier, vol. 124(1), pages 1-31, January.
  30. N. Kundan Kishor & Evan F. Koenig, 2009. "VAR Estimation and Forecasting When Data Are Subject to Revision," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(2), pages 181-190, July.
  31. Lovell, Michael C, 1983. "Data Mining," The Review of Economics and Statistics, MIT Press, vol. 65(1), pages 1-12, February.
  32. Calhoun, Gray, 2014. "Out-Of-Sample Comparisons of Overfit Models," Staff General Research Papers 32462, Iowa State University, Department of Economics.
  33. Howrey, E Philip, 1978. "The Use of Preliminary Data in Econometric Forecasting," The Review of Economics and Statistics, MIT Press, vol. 60(2), pages 193-200, May.
  34. Inoue, Atsushi & Rossi, Barbara, 2005. "Recursive Predictability Tests for Real-Time Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 336-345, July.
  35. Francis X. Diebold & Glenn D. Rudebusch, 1989. "Forecasting output with the composite leading index: an ex ante analysis," Finance and Economics Discussion Series 90, Board of Governors of the Federal Reserve System (U.S.).
  36. Denton, Frank T, 1985. "Data Mining as an Industry," The Review of Economics and Statistics, MIT Press, vol. 67(1), pages 124-27, February.
  37. Lutz Kilian, 1998. "Small-Sample Confidence Intervals For Impulse Response Functions," The Review of Economics and Statistics, MIT Press, vol. 80(2), pages 218-230, May.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:fip:fedlwp:2011-025. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Anna Xiao)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.