This file is part of IDEAS, which uses RePEc data


[ Papers | Articles | Software | Books | Chapters | Authors | Institutions | JEL Classification | NEP reports | Search | New papers by email | Author registration | Rankings | Volunteers | FAQ | Blog | Help! ]

Forecast-based model selection in the presence of structural breaks

Author info | Abstract | Publisher info | Download info | Related research | Statistics
Author Info
Todd E. Clark
Michael W. McCracken

Additional information is available for the following registered author(s):

Abstract

This paper presents analytical, Monte Carlo, and empirical evidence on the effects of structural breaks on tests for equal forecast accuracy and forecast encompassing. The forecasts are generated from two parametric, linear models that are nested under the null. The alternative hypotheses allow a causal relationship that is subject to breaks during the sample. With this framework, we show that in-sample explanatory power is readily found because the usual F-test will indicate causality if it existed for any portion of the sample. Out-of-sample predictive power can be harder to find because the results of out-of-sample tests are highly dependent on the timing of the predictive ability. Moreover, out-of-sample predictive power is harder to find with some tests than with others: the power of F-type tests of equal forecast accuracy and encompassing often dominates that of the more commonly-used t-type alternatives. Overall, out-of-sample tests are effective at revealing whether one variable has predictive power for another at the end of the sample. Based on these results and additional evidence from two empirical applications, we conclude that structural breaks can explain why researchers often find evidence of in-sample, but not out-of-sample, predictive content.

Download Info
To download:

If you experience problems downloading a file, check if you have the proper application to view it first. Information about this may be contained in the File-Format links below. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.kc.frb.org/Publicat/Reswkpap/pdf/rwp02-05.pdf
File Format: application/pdf
File Function:
Download Restriction: no

Publisher Info
Paper provided by Federal Reserve Bank of Kansas City in its series Research Working Paper with number RWP 02-05.

Download reference. The following formats are available: HTML (with abstract), plain text (with abstract), BibTeX, RIS (EndNote, RefMan, ProCite), ReDIF
Length:
Date of creation: 2002
Date of revision:
Handle: RePEc:fip:fedkrw:rwp02-05

Contact details of provider:
Postal: 1 Memorial Drive, Kansas City, MO 64198-0001
Phone: (816) 881-2254
Email:
Web page: http://www.kansascityfed.org/
More information through EDIRC

Order Information:
Email:
Web: http://app.ny.frb.org/cfpicnic/frame1.cfm

For technical questions regarding this item, or to correct its listing, contact: (Diane Rosenberger).

Related research
Keywords: Forecasting;

Other versions of this item:

This paper has been announced in the following NEP Reports: References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
  1. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November. [Downloadable!] (restricted)
    Other versions:
  2. Kuo, Biing-Shen & Mikkola, Anne, 1999. "Re-examining long-run purchasing power parity," Journal of International Money and Finance, Elsevier, vol. 18(2), pages 251-266, February. [Downloadable!] (restricted)
  3. Todd E. Clark & Michael W. McCracken, 2001. "Evaluating long-horizon forecasts," Research Working Paper RWP 01-14, Federal Reserve Bank of Kansas City. [Downloadable!]
  4. Ghysels, Eric & Hall, Alastair, 1990. "A Test for Structural Stability of Euler Conditions Parameters Estimated via the Generalized Method of Moments Estimator," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 31(2), pages 355-64, May. [Downloadable!] (restricted)
    Other versions:
  5. James H. Stock & Mark W. Watson, 2002. "Has the Business Cycle Changed and Why?," NBER Working Papers 9127, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
    Other versions:
  6. Rudebusch, Glenn D, 1993. "The Uncertain Unit Root in Real GNP," American Economic Review, American Economic Association, vol. 83(1), pages 264-72, March. [Downloadable!] (restricted)
    Other versions:
  7. Andrews, Donald W K & Ploberger, Werner, 1994. "Optimal Tests When a Nuisance Parameter Is Present Only under the Alternative," Econometrica, Econometric Society, vol. 62(6), pages 1383-1414, November. [Downloadable!] (restricted)
    Other versions:
  8. Meese, Richard A. & Rogoff, Kenneth, 1983. "Empirical exchange rate models of the seventies : Do they fit out of sample?," Journal of International Economics, Elsevier, vol. 14(1-2), pages 3-24, February. [Downloadable!] (restricted)
  9. repec:cup:etheor:v:8:y:1992:i:4:p:489-500 is not listed on IDEAS
  10. Lutz Kilian & Mark P. Taylor, 2001. "Why is it so difficult to beat the random walk forecast of exchange rates," Working Paper Series 088, European Central Bank. [Downloadable!]
    Other versions:
  11. Pesaran, M. Hashem & Timmermann, Allan, 2002. "Market timing and return prediction under model instability," Journal of Empirical Finance, Elsevier, vol. 9(5), pages 495-510, December. [Downloadable!] (restricted)
    Other versions:
  12. Hendry, David F., 2000. "On detectable and non-detectable structural change," Structural Change and Economic Dynamics, Elsevier, vol. 11(1-2), pages 45-65, July. [Downloadable!] (restricted)
  13. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
  14. Estrella, Arturo & Hardouvelis, Gikas A, 1991. " The Term Structure as a Predictor of Real Economic Activity," Journal of Finance, American Finance Association, vol. 46(2), pages 555-76, June. [Downloadable!] (restricted)
    Other versions:
  15. Arturo Estrella & Anthony P. Rodrigues & Sebastian Schich, 2003. "How Stable is the Predictive Power of the Yield Curve? Evidence from Germany and the United States," The Review of Economics and Statistics, MIT Press, vol. 85(3), pages 629-644, 07. [Downloadable!] (restricted)
    Other versions:
  16. Croushore, Dean & Stark, Tom, 2001. "A real-time data set for macroeconomists," Journal of Econometrics, Elsevier, vol. 105(1), pages 111-130, November. [Downloadable!] (restricted)
    Other versions:
  17. Bruce E. Hansen, 1995. "Approximate Asymptotic P-Values for Structural Change Tests," Boston College Working Papers in Economics 297., Boston College Department of Economics. [Downloadable!]
    Other versions:
  18. Chong, Yock Y & Hendry, David F, 1986. "Econometric Evaluation of Linear Macro-Economic Models," Review of Economic Studies, Blackwell Publishing, vol. 53(4), pages 671-90, August. [Downloadable!] (restricted)
  19. James H. Stock & Mark W. Watson, 2001. "Forecasting output and inflation: the role of asset prices," Proceedings, Federal Reserve Bank of San Francisco, issue Mar. [Downloadable!]
    Other versions:
  20. Hansen, Bruce E., 2000. "Testing for structural change in conditional models," Journal of Econometrics, Elsevier, vol. 97(1), pages 93-115, July. [Downloadable!] (restricted)
    Other versions:
  21. West, Kenneth D, 2001. "Tests for Forecast Encompassing When Forecasts Depend on Estimated Regression Parameters," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(1), pages 29-33, January.
  22. Marcellino, Massimiliano, 2002. "Instability and Non-Linearity in the EMU," CEPR Discussion Papers 3312, C.E.P.R. Discussion Papers. [Downloadable!] (restricted)
    Other versions:
  23. Harvey, David I & Leybourne, Stephen J & Newbold, Paul, 1998. "Tests for Forecast Encompassing," Journal of Business & Economic Statistics, American Statistical Association, vol. 16(2), pages 254-59, April.
  24. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-84, September. [Downloadable!] (restricted)
    Other versions:
  25. Inoue, Atsushi & Kilian, Lutz, 2002. "In-Sample or Out-of-Sample Tests of Predictability: Which One Should We Use?," CEPR Discussion Papers 3671, C.E.P.R. Discussion Papers. [Downloadable!] (restricted)
    Other versions:
  26. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-38, May. [Downloadable!] (restricted)
  27. West, Kenneth D & McCracken, Michael W, 1998. "Regression-Based Tests of Predictive Ability," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 817-40, November.
    Other versions:
  28. Norman R. Swanson, 2000. "An Out of Sample Test for Granger Causality," Econometric Society World Congress 2000 Contributed Papers 0362, Econometric Society. [Downloadable!]
  29. Michael P. Clements & David F. Hendry, 2001. "Forecasting Non-Stationary Economic Time Series," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262531895.
  30. Inoue, Atsushi & Kilian, Lutz, 2003. "On the Selection of Forecasting Models," CEPR Discussion Papers 3809, C.E.P.R. Discussion Papers. [Downloadable!] (restricted)
    Other versions:
  31. Rossi, Barbara, 2002. "Optimal Tests for Nested Model Selection with Underlying Parameter Instability," Working Papers 02-05, Duke University, Department of Economics. [Downloadable!]
    Other versions:
  32. Jushan Bai & Pierre Perron, 1998. "Estimating and Testing Linear Models with Multiple Structural Changes," Econometrica, Econometric Society, vol. 66(1), pages 47-78, January.
    Other versions:
  33. Diebold, Francis X & Mariano, Roberto S, 1995. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(3), pages 253-63, July.
    Other versions:
  34. Andrews, Donald W K, 1993. "Tests for Parameter Instability and Structural Change with Unknown Change Point," Econometrica, Econometric Society, vol. 61(4), pages 821-56, July. [Downloadable!] (restricted)
    Other versions:
Full references

Cited by:
(explanations, Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.)

  1. Lutz Kilian & Atsushi Inoue, 2002. "In-Sample or out-of-sample tests of predictability: which one should we use?," Working Paper Series 195, European Central Bank. [Downloadable!]
    Other versions:
  2. Jorge Selaive & Vicente Tuesta, 2004. "Can Fluctuations in the Consumption-Wealth Ratio Help to Predict Exchange Rates?," International Finance 0404014, EconWPA. [Downloadable!]
    Other versions:
  3. Todd E. Clark & Michael W. McCracken, 2003. "The predictive content of the output gap for inflation : resolving in-sample and out-of-sample evidence," Research Working Paper RWP 03-06, Federal Reserve Bank of Kansas City. [Downloadable!]
    Other versions:
  4. Valentina Corradi & Norman Swanson, 2004. "Predective Density and Conditional Confidence Interval Accuracy Tests," Departmental Working Papers 200423, Rutgers University, Department of Economics. [Downloadable!]
    Other versions:
Statistics
Access and download statistics

Did you know? All top Economics journals are listed on RePEc.

This page was last updated on 2009-11-18.

This information is provided to you by IDEAS at the Department of Economics, College of Liberal Arts and Sciences, University of Connecticut using RePEc data on a server sponsored by the Society for Economic Dynamics.