Advanced Search
MyIDEAS: Login to save this paper or follow this series

Advances in forecast evaluation

Contents:

Author Info

  • Todd Clark
  • Michael W. McCracken

Abstract

This paper surveys recent developments in the evaluation of point forecasts. Taking West’s (2006) survey as a starting point, we briefly cover the state of the literature as of the time of West’s writing. We then focus on recent developments, including advancements in the evaluation of forecasts at the population level (based on true, unknown model coefficients), the evaluation of forecasts in the finite sample (based on estimated model coefficients), and the evaluation of conditional versus unconditional forecasts. We present original results in a few subject areas: the optimization of power in determining the split of a sample into in-sample and out-of-sample portions; whether the accuracy of inference in evaluation of multistep forecasts can be improved with the judicious choice of HAC estimator (it can); and the extension of West’s (1996) theory results for population-level, unconditional forecast evaluation to the case of conditional forecast evaluation.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://www.clevelandfed.org/research/Workpaper/2011/wp1120.pdf
Download Restriction: no

Bibliographic Info

Paper provided by Federal Reserve Bank of Cleveland in its series Working Paper with number 1120.

as in new window
Length:
Date of creation: 2011
Date of revision:
Handle: RePEc:fip:fedcwp:1120

Contact details of provider:
Postal: 1455 East 6th St., Cleveland OH 44114
Phone: 216.579.2000
Web page: http://www.clevelandfed.org/
More information through EDIRC

Order Information:
Email:

Related research

Keywords: Forecasting ; Time-series analysis;

Other versions of this item:

This paper has been announced in the following NEP Reports:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Orphanides, Athanasios & van Norden, Simon, 2005. "The Reliability of Inflation Forecasts Based on Output Gap Estimates in Real Time," Journal of Money, Credit and Banking, Blackwell Publishing, Blackwell Publishing, vol. 37(3), pages 583-601, June.
  2. Yu-chin Chen & Kenneth Rogoff & Barbara Rossi, 2008. "Can Exchange Rates Forecast Commodity Prices?," Working Papers, University of Washington, Department of Economics UWEC-2008-11-FC, University of Washington, Department of Economics, revised Oct 2009.
  3. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, American Statistical Association, vol. 20(1), pages 134-44, January.
  4. de Jong, Robert M. & Davidson, James, 2000. "The Functional Central Limit Theorem And Weak Convergence To Stochastic Integrals I," Econometric Theory, Cambridge University Press, Cambridge University Press, vol. 16(05), pages 621-642, October.
  5. Todd E. Clark & Michael W. McCracken, 1999. "Tests of equal forecast accuracy and encompassing for nested models," Research Working Paper, Federal Reserve Bank of Kansas City 99-11, Federal Reserve Bank of Kansas City.
  6. Stanislav Anatolyev, 2007. "Inference about predictive ability when there are many predictors," Working Papers w0096, Center for Economic and Financial Research (CEFIR).
  7. Todd E. Clark & Kenneth D. West, 2005. "Approximately normal tests for equal predictive accuracy in nested models," Research Working Paper, Federal Reserve Bank of Kansas City RWP 05-05, Federal Reserve Bank of Kansas City.
  8. Raffella Giacomini & Barbara Rossi, 2005. "Detecting and Predicting Forecast Breakdowns," UCLA Economics Working Papers, UCLA Department of Economics 845, UCLA Department of Economics.
  9. Amit Goval & Ivo Welch, 2004. "A Comprehensive Look at the Empirical Performance of Equity Premium Prediction," NBER Working Papers 10483, National Bureau of Economic Research, Inc.
  10. Barbara Rossi & Atsushi Inoue, 2011. "Out-of-Sample Forecast Tests Robust to Window Size Choice," Working Papers, Duke University, Department of Economics 11-04, Duke University, Department of Economics.
  11. Clark, Todd E. & West, Kenneth D., 2006. "Using out-of-sample mean squared prediction errors to test the martingale difference hypothesis," Journal of Econometrics, Elsevier, Elsevier, vol. 135(1-2), pages 155-186.
  12. Lovell, Michael C, 1983. "Data Mining," The Review of Economics and Statistics, MIT Press, vol. 65(1), pages 1-12, February.
  13. Ashley, R & Granger, C W J & Schmalensee, R, 1980. "Advertising and Aggregate Consumption: An Analysis of Causality," Econometrica, Econometric Society, Econometric Society, vol. 48(5), pages 1149-67, July.
  14. Busetti, Fabio & Marcucci, Juri, 2013. "Comparing forecast accuracy: A Monte Carlo investigation," International Journal of Forecasting, Elsevier, Elsevier, vol. 29(1), pages 13-27.
  15. Todd E. Clark & Michael W. McCracken, 2009. "Nested forecast model comparisons: a new approach to testing equal accuracy," Research Working Paper, Federal Reserve Bank of Kansas City RWP 09-11, Federal Reserve Bank of Kansas City.
  16. Aruoba, Boragan, 2005. "Data Revisions Are Not Well-Behaved," CEPR Discussion Papers, C.E.P.R. Discussion Papers 5271, C.E.P.R. Discussion Papers.
  17. Todd E. Clark & Michael W. McCracken, 2008. "Tests of equal predictive ability with real-time data," Working Papers, Federal Reserve Bank of St. Louis 2008-029, Federal Reserve Bank of St. Louis.
  18. Clark, Todd E. & McCracken, Michael W., 2005. "The power of tests of predictive ability in the presence of structural breaks," Journal of Econometrics, Elsevier, Elsevier, vol. 124(1), pages 1-31, January.
  19. Evan Koenig & Sheila Dolmas & Jeremy M. Piger, 2002. "The use and abuse of 'real-time' data in economic forecasting," Working Papers, Federal Reserve Bank of St. Louis 2001-015, Federal Reserve Bank of St. Louis.
  20. Denton, Frank T, 1985. "Data Mining as an Industry," The Review of Economics and Statistics, MIT Press, vol. 67(1), pages 124-27, February.
  21. Calhoun, Gray, 2014. "Out-Of-Sample Comparisons of Overfit Models," Staff General Research Papers, Iowa State University, Department of Economics 32462, Iowa State University, Department of Economics.
  22. West, Kenneth D., 1997. "Another heteroskedasticity- and autocorrelation-consistent covariance matrix estimator," Journal of Econometrics, Elsevier, Elsevier, vol. 76(1-2), pages 171-191.
  23. Harvey, David & Leybourne, Stephen & Newbold, Paul, 1997. "Testing the equality of prediction mean squared errors," International Journal of Forecasting, Elsevier, Elsevier, vol. 13(2), pages 281-291, June.
  24. Fair, Ray C & Shiller, Robert J, 1989. "The Informational Context of Ex Ante Forecasts," The Review of Economics and Statistics, MIT Press, vol. 71(2), pages 325-31, May.
  25. McCracken, Michael W., 2007. "Asymptotics for out of sample tests of Granger causality," Journal of Econometrics, Elsevier, Elsevier, vol. 140(2), pages 719-752, October.
  26. McCracken, Michael W., 2004. "Parameter estimation and tests of equal forecast accuracy between non-nested models," International Journal of Forecasting, Elsevier, Elsevier, vol. 20(3), pages 503-514.
  27. Todd E. Clark & Michael W. McCracken, 2010. "Testing for unconditional predictive ability," Working Papers, Federal Reserve Bank of St. Louis 2010-031, Federal Reserve Bank of St. Louis.
  28. N. Kundan Kishor & Evan F. Koenig, 2009. "VAR Estimation and Forecasting When Data Are Subject to Revision," Journal of Business & Economic Statistics, Taylor & Francis Journals, Taylor & Francis Journals, vol. 30(2), pages 181-190, July.
  29. Rossi, Barbara & Inoue, Atsushi, 2003. "Recursive Predictability Tests for Real-Time Data," Working Papers, Duke University, Department of Economics 03-24, Duke University, Department of Economics.
  30. Todd Clark & Michael McCracken, 2005. "Evaluating Direct Multistep Forecasts," Econometric Reviews, Taylor & Francis Journals, Taylor & Francis Journals, vol. 24(4), pages 369-404.
  31. Mc Cracken, Michael W., 2000. "Robust out-of-sample inference," Journal of Econometrics, Elsevier, Elsevier, vol. 99(2), pages 195-223, December.
  32. Kilian, Lutz, 1999. "Exchange Rates and Monetary Fundamentals: What Do We Learn from Long-Horizon Regressions?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., John Wiley & Sons, Ltd., vol. 14(5), pages 491-510, Sept.-Oct.
  33. Hansen, Bruce E., 1992. "Convergence to Stochastic Integrals for Dependent Heterogeneous Processes," Econometric Theory, Cambridge University Press, Cambridge University Press, vol. 8(04), pages 489-500, December.
  34. Howrey, E Philip, 1978. "The Use of Preliminary Data in Econometric Forecasting," The Review of Economics and Statistics, MIT Press, vol. 60(2), pages 193-200, May.
  35. Clements, Michael P. & Galvão, Ana Beatriz, 2010. "Real-time Forecasting of Inflation and Output Growth in the Presence of Data Revisions," The Warwick Economics Research Paper Series (TWERPS) 953, University of Warwick, Department of Economics.
  36. Lutz Kilian, 1998. "Small-Sample Confidence Intervals For Impulse Response Functions," The Review of Economics and Statistics, MIT Press, vol. 80(2), pages 218-230, May.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. Todd E.Clark & Taeyoung Doh, 2011. "A Bayesian evaluation of alternative models of trend inflation," Research Working Paper, Federal Reserve Bank of Kansas City RWP 11-16, Federal Reserve Bank of Kansas City.
  2. Francis X. Diebold, 2012. "Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold-Mariano Tests," PIER Working Paper Archive 12-035, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
  3. Inoue, Atsushi & Rossi, Barbara, 2011. "Out-of-Sample Forecast Tests Robust to the Choice of Window Size," CEPR Discussion Papers, C.E.P.R. Discussion Papers 8542, C.E.P.R. Discussion Papers.
  4. Francesco Ravazzolo & Philip Rothman, 2013. "Oil and U.S. GDP: A Real‐Time Out‐of‐Sample Examination," Journal of Money, Credit and Banking, Blackwell Publishing, Blackwell Publishing, vol. 45(2-3), pages 449-463, 03.
  5. Gregory R. Duffee, 2012. "Forecasting interest rates," Economics Working Paper Archive, The Johns Hopkins University,Department of Economics 599, The Johns Hopkins University,Department of Economics.
  6. Galvão, Ana Beatriz, 2013. "Changes in predictive ability with mixed frequency data," International Journal of Forecasting, Elsevier, Elsevier, vol. 29(3), pages 395-410.
  7. Busetti, Fabio & Marcucci, Juri, 2013. "Comparing forecast accuracy: A Monte Carlo investigation," International Journal of Forecasting, Elsevier, Elsevier, vol. 29(1), pages 13-27.
  8. Peter Reinhard HANSEN & Allan TIMMERMANN, 2012. "Choice of Sample Split in Out-of-Sample Forecast Evaluation," Economics Working Papers, European University Institute ECO2012/10, European University Institute.
  9. Arai, Natsuki, 2014. "Using forecast evaluation to improve the accuracy of the Greenbook forecast," International Journal of Forecasting, Elsevier, Elsevier, vol. 30(1), pages 12-19.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:fip:fedcwp:1120. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Lee Faulhaber).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.