IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Login to save this paper or follow this series

Advances in forecast evaluation

  • Todd Clark
  • Michael W. McCracken

This paper surveys recent developments in the evaluation of point forecasts. Taking West’s (2006) survey as a starting point, we briefly cover the state of the literature as of the time of West’s writing. We then focus on recent developments, including advancements in the evaluation of forecasts at the population level (based on true, unknown model coefficients), the evaluation of forecasts in the finite sample (based on estimated model coefficients), and the evaluation of conditional versus unconditional forecasts. We present original results in a few subject areas: the optimization of power in determining the split of a sample into in-sample and out-of-sample portions; whether the accuracy of inference in evaluation of multistep forecasts can be improved with the judicious choice of HAC estimator (it can); and the extension of West’s (1996) theory results for population-level, unconditional forecast evaluation to the case of conditional forecast evaluation.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: https://www.clevelandfed.org/en/Newsroom%20and%20Events/Publications/Working%20Papers/~/media/6EFFBE276136416BBC4DF11D0AF2E659.ashx
File Function: Full text
Download Restriction: no

Paper provided by Federal Reserve Bank of Cleveland in its series Working Paper with number 1120.

as
in new window

Length:
Date of creation: 2011
Date of revision:
Handle: RePEc:fip:fedcwp:1120
Contact details of provider: Postal: 1455 East 6th St., Cleveland OH 44114
Phone: 216.579.2000
Web page: http://www.clevelandfed.org/

More information through EDIRC

Order Information: Email:


References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Kenneth D. West, 1995. "Another Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimator," NBER Technical Working Papers 0183, National Bureau of Economic Research, Inc.
  2. Yu-chin Chen & Kenneth Rogoff & Barbara Rossi, 2008. "Can Exchange Rates Forecast Commodity Prices?," Working Papers UWEC-2008-11-FC, University of Washington, Department of Economics, revised Oct 2009.
  3. Davidson, James & de Jong, Robert M., 2000. "The Functional Central Limit Theorem And Weak Convergence To Stochastic Integrals Ii," Econometric Theory, Cambridge University Press, vol. 16(05), pages 643-666, October.
  4. Ivo Welch & Amit Goyal, 2008. "A Comprehensive Look at The Empirical Performance of Equity Premium Prediction," Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1455-1508, July.
  5. Kilian, Lutz, 1999. "Exchange Rates and Monetary Fundamentals: What Do We Learn from Long-Horizon Regressions?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 14(5), pages 491-510, Sept.-Oct.
  6. Clark, Todd E. & West, Kenneth D., 2006. "Using out-of-sample mean squared prediction errors to test the martingale difference hypothesis," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 155-186.
  7. Todd E. Clark & Michael W. McCracken, 2007. "Tests of equal predictive ability with real-time data," Research Working Paper RWP 07-06, Federal Reserve Bank of Kansas City.
  8. Clark, Todd E. & McCracken, Michael W., 2005. "The power of tests of predictive ability in the presence of structural breaks," Journal of Econometrics, Elsevier, vol. 124(1), pages 1-31, January.
  9. Hansen, Bruce E., 1992. "Convergence to Stochastic Integrals for Dependent Heterogeneous Processes," Econometric Theory, Cambridge University Press, vol. 8(04), pages 489-500, December.
  10. Raffella Giacomini & Barbara Rossi, 2005. "Detecting and Predicting Forecast Breakdowns," UCLA Economics Working Papers 845, UCLA Department of Economics.
  11. Ashley, R & Granger, C W J & Schmalensee, R, 1980. "Advertising and Aggregate Consumption: An Analysis of Causality," Econometrica, Econometric Society, vol. 48(5), pages 1149-67, July.
  12. Stanislav Anatolyev, 2007. "Inference about predictive ability when there are many predictors," Working Papers w0096, Center for Economic and Financial Research (CEFIR).
  13. Calhoun, Gray, 2014. "Out-Of-Sample Comparisons of Overfit Models," Staff General Research Papers 32462, Iowa State University, Department of Economics.
  14. N. Kundan Kishor & Evan F. Koenig, 2009. "VAR Estimation and Forecasting When Data Are Subject to Revision," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(2), pages 181-190, July.
  15. Lovell, Michael C, 1983. "Data Mining," The Review of Economics and Statistics, MIT Press, vol. 65(1), pages 1-12, February.
  16. Francis X. Diebold & Glenn D. Rudebusch, 1989. "Forecasting output with the composite leading index: an ex ante analysis," Finance and Economics Discussion Series 90, Board of Governors of the Federal Reserve System (U.S.).
  17. Todd E. Clark & Michael W. McCracken, 2009. "Nested forecast model comparisons: a new approach to testing equal accuracy," Research Working Paper RWP 09-11, Federal Reserve Bank of Kansas City.
  18. Inoue, Atsushi & Rossi, Barbara, 2005. "Recursive Predictability Tests for Real-Time Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 336-345, July.
  19. Busetti, Fabio & Marcucci, Juri, 2013. "Comparing forecast accuracy: A Monte Carlo investigation," International Journal of Forecasting, Elsevier, vol. 29(1), pages 13-27.
  20. Howrey, E Philip, 1978. "The Use of Preliminary Data in Econometric Forecasting," The Review of Economics and Statistics, MIT Press, vol. 60(2), pages 193-200, May.
  21. Orphanides, Athanasios & van Norden, Simon, 2005. "The Reliability of Inflation Forecasts Based on Output Gap Estimates in Real Time," CEPR Discussion Papers 4830, C.E.P.R. Discussion Papers.
  22. Lutz Kilian, 1998. "Small-Sample Confidence Intervals For Impulse Response Functions," The Review of Economics and Statistics, MIT Press, vol. 80(2), pages 218-230, May.
  23. Denton, Frank T, 1985. "Data Mining as an Industry," The Review of Economics and Statistics, MIT Press, vol. 67(1), pages 124-27, February.
  24. Todd E. Clark & Michael W. McCracken, 1999. "Tests of equal forecast accuracy and encompassing for nested models," Research Working Paper 99-11, Federal Reserve Bank of Kansas City.
  25. Evan Koenig & Sheila Dolmas & Jeremy M. Piger, 2002. "The use and abuse of 'real-time' data in economic forecasting," Working Papers 2001-015, Federal Reserve Bank of St. Louis.
  26. Ray C. Fair & Robert J. Shiller, 1988. "The Informational Content of Ex Ante Forecasts," NBER Working Papers 2503, National Bureau of Economic Research, Inc.
  27. Diebold, Francis X & Mariano, Roberto S, 1995. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(3), pages 253-63, July.
  28. Clark, Todd E. & West, Kenneth D., 2007. "Approximately normal tests for equal predictive accuracy in nested models," Journal of Econometrics, Elsevier, vol. 138(1), pages 291-311, May.
  29. Clements, Michael P. & Galvão, Ana Beatriz, 2010. "Real-time Forecasting of Inflation and Output Growth in the Presence of Data Revisions," The Warwick Economics Research Paper Series (TWERPS) 953, University of Warwick, Department of Economics.
  30. Aruoba, Boragan, 2005. "Data Revisions Are Not Well-Behaved," CEPR Discussion Papers 5271, C.E.P.R. Discussion Papers.
  31. McCracken, Michael W., 2004. "Parameter estimation and tests of equal forecast accuracy between non-nested models," International Journal of Forecasting, Elsevier, vol. 20(3), pages 503-514.
  32. Mc Cracken, Michael W., 2000. "Robust out-of-sample inference," Journal of Econometrics, Elsevier, vol. 99(2), pages 195-223, December.
  33. Barbara Rossi & Atsushi Inoue, 2011. "Out-of-Sample Forecast Tests Robust to Window Size Choice," Working Papers 11-04, Duke University, Department of Economics.
  34. Harvey, David & Leybourne, Stephen & Newbold, Paul, 1997. "Testing the equality of prediction mean squared errors," International Journal of Forecasting, Elsevier, vol. 13(2), pages 281-291, June.
  35. Todd Clark & Michael McCracken, 2005. "Evaluating Direct Multistep Forecasts," Econometric Reviews, Taylor & Francis Journals, vol. 24(4), pages 369-404.
  36. McCracken, Michael W., 2007. "Asymptotics for out of sample tests of Granger causality," Journal of Econometrics, Elsevier, vol. 140(2), pages 719-752, October.
  37. Todd E. Clark & Michael W. McCracken, 2010. "Testing for unconditional predictive ability," Working Papers 2010-031, Federal Reserve Bank of St. Louis.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:fip:fedcwp:1120. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Lee Faulhaber)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.