Advanced Search
MyIDEAS: Login

Forecast evaluation of small nested model sets

Contents:

Author Info

  • Hubrich, Kirstin
  • West, Kenneth D.

Abstract

We propose two new procedures for comparing the mean squared prediction error (MSPE) of a benchmark model to the MSPEs of a small set of alternative models that nest the benchmark. Our procedures compare the bench-mark to all the alternative models simultaneously rather than sequentially, and do not require re-estimation of models as part of a bootstrap procedure. Both procedures adjust MSPE differences in accordance with Clark and West (2007); one procedure then examines the maximum t-statistic, the other computes a chi-squared statistic. Our simulations examine the proposed procedures and two existing procedures that do not adjust the MSPE differences: a chi-squared statistic, and White’s (2000) reality check. In these simulations, the two statistics that adjust MSPE differences have most accurate size, and the procedure that looks at the maximum t-statistic has best power. We illustrate, our procedures by comparing forecasts of different models for U.S. inflation. JEL Classification: C32, C53, E37

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://www.ecb.europa.eu/pub/pdf/scpwps/ecbwp1030.pdf
Download Restriction: no

Bibliographic Info

Paper provided by European Central Bank in its series Working Paper Series with number 1030.

as in new window
Length:
Date of creation: Mar 2009
Date of revision:
Handle: RePEc:ecb:ecbwps:20091030

Contact details of provider:
Postal: Postfach 16 03 19, Frankfurt am Main, Germany
Phone: +49 69 1344 0
Fax: +49 69 1344 6000
Web page: http://www.ecb.europa.eu/home/html/index.en.html
More information through EDIRC

Order Information:
Postal: Press and Information Division, European Central Bank, Kaiserstrasse 29, 60311 Frankfurt am Main, Germany
Email:

Related research

Keywords: Inflation forecasting; multiple model comparisons; Out-of-Sample; prediction; testing;

Other versions of this item:

Find related papers by JEL classification:

This paper has been announced in the following NEP Reports:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Giacomini, Raffaella & White, Halbert, 2003. "Tests of Conditional Predictive Ability," University of California at San Diego, Economics Working Paper Series qt5jk0j5jh, Department of Economics, UC San Diego.
  2. Kenneth D. West & Hali J. Edison & Dongchul Cho, 1992. "A Utility Based Comparison of Some Models of Exchange Rate Volatility," NBER Technical Working Papers 0128, National Bureau of Economic Research, Inc.
  3. Orphanides, Athanasios & van Norden, Simon, 2005. "The Reliability of Inflation Forecasts Based on Output Gap Estimates in Real Time," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 37(3), pages 583-601, June.
  4. James H. Stock & Mark W. Watson, 2007. "Why Has U.S. Inflation Become Harder to Forecast?," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 39(s1), pages 3-33, 02.
  5. Giorgio Valente & Daniel Thornton & Lucio Sarno, 2004. "Federal Funds Rate Prediction," Working Papers wp04-12, Warwick Business School, Finance Group.
  6. Todd E. Clark & Kenneth D. West, 2004. "Using out-of-sample mean squared prediction errors to test the Martingale difference hypothesis," Research Working Paper RWP 04-03, Federal Reserve Bank of Kansas City.
  7. West, Kenneth D. & Cho, Dongchul, 1995. "The predictive ability of several models of exchange rate volatility," Journal of Econometrics, Elsevier, vol. 69(2), pages 367-391, October.
  8. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-84, September.
  9. McCracken, Michael W., 2007. "Asymptotics for out of sample tests of Granger causality," Journal of Econometrics, Elsevier, vol. 140(2), pages 719-752, October.
  10. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
  11. Kirstin Hubrich & David F. Hendry, 2005. "Forecasting Aggregates by Disaggregates," Computing in Economics and Finance 2005 270, Society for Computational Economics.
  12. Hubrich, Kirstin, 2005. "Forecasting euro area inflation: Does aggregating forecasts by HICP component improve forecast accuracy?," International Journal of Forecasting, Elsevier, vol. 21(1), pages 119-136.
  13. Francis X. Diebold & Robert S. Mariano, 1994. "Comparing Predictive Accuracy," NBER Technical Working Papers 0169, National Bureau of Economic Research, Inc.
  14. Inoue, Atsushi & Kilian, Lutz, 2006. "On the selection of forecasting models," Journal of Econometrics, Elsevier, vol. 130(2), pages 273-306, February.
  15. Rapach, David E. & Wohar, Mark E., 2006. "In-sample vs. out-of-sample tests of stock return predictability in the context of data mining," Journal of Empirical Finance, Elsevier, vol. 13(2), pages 231-247, March.
  16. Hendry, David F. & Hubrich, Kirstin, 2010. "Combining disaggregate forecasts or combining disaggregate information to forecast an aggregate," Working Paper Series 1155, European Central Bank.
  17. Kenneth D. West & Todd Clark, 2006. "Approximately Normal Tests for Equal Predictive Accuracy in Nested Models," NBER Technical Working Papers 0326, National Bureau of Economic Research, Inc.
  18. Lutkepohl, Helmut, 1984. "Forecasting Contemporaneously Aggregated Vector ARMA Processes," Journal of Business & Economic Statistics, American Statistical Association, vol. 2(3), pages 201-14, July.
  19. Ashley, R & Granger, C W J & Schmalensee, R, 1980. "Advertising and Aggregate Consumption: An Analysis of Causality," Econometrica, Econometric Society, vol. 48(5), pages 1149-67, July.
  20. Andrew Atkeson & Lee E. Ohanian., 2001. "Are Phillips curves useful for forecasting inflation?," Quarterly Review, Federal Reserve Bank of Minneapolis, issue Win, pages 2-11.
  21. Todd E. Clark & Michael W. McCracken, 2010. "Reality checks and nested forecast model comparisons," Working Papers 2010-032, Federal Reserve Bank of St. Louis.
  22. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
  23. Inoue, Atsushi & Kilian, Lutz, 2002. "In-sample or out-of-sample tests of predictability: which one should we use?," Working Paper Series 0195, European Central Bank.
  24. Andreas Billmeier, 2004. "Ghostbusting: Which Output Gap Measure Really Matters?," IMF Working Papers 04/146, International Monetary Fund.
  25. Hansen, Peter Reinhard, 2005. "A Test for Superior Predictive Ability," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 365-380, October.
  26. Todd Clark & Michael McCracken, 2005. "Evaluating Direct Multistep Forecasts," Econometric Reviews, Taylor & Francis Journals, vol. 24(4), pages 369-404.
  27. David Harvey & Paul Newbold, 2000. "Tests for multiple forecast encompassing," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 15(5), pages 471-482.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. D'Amuri, Francesco/FD & Marcucci, Juri/JM, 2009. ""Google it!" Forecasting the US unemployment rate with a Google job search index," MPRA Paper 18248, University Library of Munich, Germany.
  2. Todd E. Clark & Michael W. McCracken, 2010. "Reality checks and nested forecast model comparisons," Working Papers 2010-032, Federal Reserve Bank of St. Louis.
  3. Francesco Ravazzolo & Philip Rothman, 2011. "Oil and US GDP: A Real-Time out-of Sample Examination," Working Papers 0004, Centre for Applied Macro- and Petroleum economics (CAMP), BI Norwegian Business School.
  4. Mariano, Roberto S. & Preve, Daniel, 2012. "Statistical tests for multiple forecast comparison," Journal of Econometrics, Elsevier, vol. 169(1), pages 123-130.
  5. Hendry, David F. & Hubrich, Kirstin, 2011. "Combining Disaggregate Forecasts or Combining Disaggregate Information to Forecast an Aggregate," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(2), pages 216-227.
  6. Kollmann, Robert & Zeugner, Stefan, 2012. "Leverage as a predictor for real activity and volatility," Journal of Economic Dynamics and Control, Elsevier, vol. 36(8), pages 1267-1283.
  7. Aron, Janine & Muellbauer, John, 2012. "Improving forecasting in an emerging economy, South Africa: Changing trends, long run restrictions and disaggregation," International Journal of Forecasting, Elsevier, vol. 28(2), pages 456-476.
  8. Daniel Andrés Jaimes Cárdenas & Jair Ojeda Joya, . "Reglas de Taylor y previsibilidad fuera de muestra de la tasa de cambio en Latinoamérica," Borradores de Economia 619, Banco de la Republica de Colombia.
  9. Fabio Busetti & Juri Marcucci & Giovanni Veronese, 2009. "Comparing forecast accuracy: A Monte Carlo investigation," Temi di discussione (Economic working papers) 723, Bank of Italy, Economic Research and International Relations Area.
  10. Molodtsova, Tanya & Papell, David H., 2009. "Out-of-sample exchange rate predictability with Taylor rule fundamentals," Journal of International Economics, Elsevier, vol. 77(2), pages 167-180, April.
  11. Todd E. Clark & Michael W. McCracken, 2010. "Testing for unconditional predictive ability," Working Papers 2010-031, Federal Reserve Bank of St. Louis.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:ecb:ecbwps:20091030. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Official Publications).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.