IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Login to save this article or follow this journal

Forecast evaluation of small nested model sets

  • Kirstin Hubrich

    (European Central Bank, Frankfurt, Germany)

  • Kenneth D. West

    (Department of Economics, University of Wisconsin, Madison, WI, USA)

We propose two new procedures for comparing the mean squared prediction error (MSPE) of a benchmark model to the MSPEs of a small set of alternative models that nest the benchmark. Our procedures compare the benchmark to all the alternative models simultaneously rather than sequentially, and do not require re-estimation of models as part of a bootstrap procedure. Both procedures adjust MSPE differences in accordance with Clark and West (2007); one procedure then examines the maximum t-statistic, while the other computes a chi-squared statistic. Our simulations examine the proposed procedures and two existing procedures that do not adjust the MSPE differences: a chi-squared statistic and White's (2000) reality check. In these simulations, the two statistics that adjust MSPE differences have the most accurate size, and the procedure that looks at the maximum t-statistic has the best power. We illustrate our procedures by comparing forecasts of different models for US inflation. Copyright © 2010 John Wiley & Sons, Ltd.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://hdl.handle.net/10.1002/jae.1176
File Function: Link to full text; subscription required
Download Restriction: no

File URL: http://qed.econ.queensu.ca:80/jae/2010-v25.4/
File Function: Supporting data files and programs
Download Restriction: no

Article provided by John Wiley & Sons, Ltd. in its journal Journal of Applied Econometrics.

Volume (Year): 25 (2010)
Issue (Month): 4 ()
Pages: 574-594

as
in new window

Handle: RePEc:jae:japmet:v:25:y:2010:i:4:p:574-594
Contact details of provider: Web page: http://www.interscience.wiley.com/jpages/0883-7252/

Order Information: Web: http://www3.interscience.wiley.com/jcatalog/subscribe.jsp?issn=0883-7252 Email:


References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Todd E. Clark & Michael McCracken, 1999. "Tests of Equal Forecast Accuracy and Encompassing for Nested Models," Computing in Economics and Finance 1999 1241, Society for Computational Economics.
  2. Hubrich, Kirstin, 2005. "Forecasting euro area inflation: Does aggregating forecasts by HICP component improve forecast accuracy?," International Journal of Forecasting, Elsevier, vol. 21(1), pages 119-136.
  3. West, Kenneth D, 2001. "Tests for Forecast Encompassing When Forecasts Depend on Estimated Regression Parameters," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(1), pages 29-33, January.
  4. Inoue, Atsushi & Kilian, Lutz, 2002. "In-sample or out-of-sample tests of predictability: which one should we use?," Working Paper Series 0195, European Central Bank.
  5. Raffaella Giacomini & Halbert White, 2003. "Tests of conditional predictive ability," Boston College Working Papers in Economics 572, Boston College Department of Economics.
  6. Inoue, Atsushi & Kilian, Lutz, 2003. "On the Selection of Forecasting Models," CEPR Discussion Papers 3809, C.E.P.R. Discussion Papers.
  7. Hansen, Peter Reinhard, 2005. "A Test for Superior Predictive Ability," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 365-380, October.
  8. Kenneth D. West & Dongchul Cho, 1994. "The Predictive Ability of Several Models of Exchange Rate Volatility," NBER Technical Working Papers 0152, National Bureau of Economic Research, Inc.
  9. Todd E. Clark & Kenneth D. West, 2004. "Using out-of-sample mean squared prediction errors to test the Martingale difference hypothesis," Research Working Paper RWP 04-03, Federal Reserve Bank of Kansas City.
  10. Hendry, David F. & Hubrich, Kirstin, 2010. "Combining disaggregate forecasts or combining disaggregate information to forecast an aggregate," Working Paper Series 1155, European Central Bank.
  11. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-44, January.
  12. James H. Stock & Mark W. Watson, 2006. "Why Has U.S. Inflation Become Harder to Forecast?," NBER Working Papers 12324, National Bureau of Economic Research, Inc.
  13. Sarno, Lucio & Thornton, Daniel L & Valente, Giorgio, 2005. "Federal Funds Rate Prediction," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 37(3), pages 449-71, June.
  14. Athanasios Orphanides & Simon van Norden, 2003. "The Reliability of Inflation Forecasts Based on Output Gap Estimates in Real Time," CIRANO Working Papers 2003s-01, CIRANO.
  15. Clark, Todd E. & West, Kenneth D., 2007. "Approximately normal tests for equal predictive accuracy in nested models," Journal of Econometrics, Elsevier, vol. 138(1), pages 291-311, May.
  16. David Harvey & Paul Newbold, 2000. "Tests for multiple forecast encompassing," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 15(5), pages 471-482.
  17. McCracken, Michael W., 2007. "Asymptotics for out of sample tests of Granger causality," Journal of Econometrics, Elsevier, vol. 140(2), pages 719-752, October.
  18. West, K.D., 1994. "Asymptotic Inference About Predictive Ability," Working papers 9417, Wisconsin Madison - Social Systems.
  19. Yongmiao Hong & Tae-Hwy Lee, 2003. "Inference on Predictability of Foreign Exchange Rates via Generalized Spectrum and Nonlinear Time Series Models," The Review of Economics and Statistics, MIT Press, vol. 85(4), pages 1048-1062, November.
  20. Rapach, David E. & Wohar, Mark E., 2006. "In-sample vs. out-of-sample tests of stock return predictability in the context of data mining," Journal of Empirical Finance, Elsevier, vol. 13(2), pages 231-247, March.
  21. Todd E. Clark & Michael W. McCracken, 2010. "Reality checks and nested forecast model comparisons," Working Papers 2010-032, Federal Reserve Bank of St. Louis.
  22. Kirstin Hubrich & David F. Hendry, 2005. "Forecasting Aggregates by Disaggregates," Computing in Economics and Finance 2005 270, Society for Computational Economics.
  23. Ashley, R & Granger, C W J & Schmalensee, R, 1980. "Advertising and Aggregate Consumption: An Analysis of Causality," Econometrica, Econometric Society, vol. 48(5), pages 1149-67, July.
  24. Todd Clark & Michael McCracken, 2005. "Evaluating Direct Multistep Forecasts," Econometric Reviews, Taylor & Francis Journals, vol. 24(4), pages 369-404.
  25. West, Kenneth D. & Edison, Hali J. & Cho, Dongchul, 1993. "A utility-based comparison of some models of exchange rate volatility," Journal of International Economics, Elsevier, vol. 35(1-2), pages 23-45, August.
  26. Lutkepohl, Helmut, 1984. "Forecasting Contemporaneously Aggregated Vector ARMA Processes," Journal of Business & Economic Statistics, American Statistical Association, vol. 2(3), pages 201-14, July.
  27. Andrew Atkeson & Lee E. Ohanian., 2001. "Are Phillips curves useful for forecasting inflation?," Quarterly Review, Federal Reserve Bank of Minneapolis, issue Win, pages 2-11.
  28. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
  29. Andreas Billmeier, 2004. "Ghostbusting; Which Output Gap Measure Really Matters?," IMF Working Papers 04/146, International Monetary Fund.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:jae:japmet:v:25:y:2010:i:4:p:574-594. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Wiley-Blackwell Digital Licensing)

or (Christopher F. Baum)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.