Advanced Search
MyIDEAS: Login

Detecting and predicting forecast breakdowns

Contents:

Author Info

Abstract

We propose a theoretical framework for assessing whether a forecast model estimated over one period can provide good forecasts over a subsequent period. We formalize this idea by defining a forecast breakdown as a situation in which the out-of-sample performance of the model, judged by some loss function, is significantly worse than its in-sample performance. Our framework, which is valid under general conditions, can be used not only to detect past forecast breakdowns but also to predict future ones. We show that main causes of forecast breakdowns are instabilities in the data generating process and relate the properties of our forecast breakdown test to those of existing structural break tests. The empirical application finds evidence of a forecast breakdown in the Phillips’ curve forecasts of U.S. inflation, and links it to inflation volatility and to changes in the monetary policy reaction function of the Fed. JEL Classification: C22; C52; C53.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://www.ecb.europa.eu/pub/pdf/scpwps/ecbwp638.pdf
Download Restriction: no

Bibliographic Info

Paper provided by European Central Bank in its series Working Paper Series with number 638.

as in new window
Length: 49 pages
Date of creation: Jun 2006
Date of revision:
Handle: RePEc:ecb:ecbwps:20060638

Contact details of provider:
Postal: Postfach 16 03 19, Frankfurt am Main, Germany
Phone: +49 69 1344 0
Fax: +49 69 1344 6000
Web page: http://www.ecb.europa.eu/home/html/index.en.html
More information through EDIRC

Order Information:
Postal: Press and Information Division, European Central Bank, Kaiserstrasse 29, 60311 Frankfurt am Main, Germany
Email:

Related research

Keywords: Structural change; forecast evaluation; forecast rationality testing; in-sample evaluation; out-of-sample evaluation.;

Other versions of this item:

Find related papers by JEL classification:

This paper has been announced in the following NEP Reports:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Clarida, Richard & Galí, Jordi & Gertler, Mark, 1998. "Monetary Policy Rules and Macroeconomic Stability: Evidence and Some Theory," CEPR Discussion Papers 1908, C.E.P.R. Discussion Papers.
  2. Amit Goval & Ivo Welch, 2004. "A Comprehensive Look at the Empirical Performance of Equity Premium Prediction," NBER Working Papers 10483, National Bureau of Economic Research, Inc.
  3. Ghysels, Eric & Hall, Alastair, 1990. "A Test for Structural Stability of Euler Conditions Parameters Estimated via the Generalized Method of Moments Estimator," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 31(2), pages 355-64, May.
  4. Perron, P. & Bai, J., 1995. "Estimating and Testing Linear Models with Multiple Structural Changes," Cahiers de recherche 9552, Centre interuniversitaire de recherche en économie quantitative, CIREQ.
  5. Myles Callan & Eric Ghysels & Norman R. Swanson, 1998. "Monetary Policy Rules with Model and Data Uncertainty," CIRANO Working Papers 98s-40, CIRANO.
  6. Pesaran, M.H. & Pettenuzzo, D. & Timmermann, A., 2004. "‘Forecasting Time Series Subject to Multiple Structural Breaks’," Cambridge Working Papers in Economics 0433, Faculty of Economics, University of Cambridge.
  7. Rossi, Barbara, 2005. "Optimal Tests For Nested Model Selection With Underlying Parameter Instability," Econometric Theory, Cambridge University Press, vol. 21(05), pages 962-990, October.
  8. Douglas Staiger & James H. Stock & Mark W. Watson, 1997. "The NAIRU, Unemployment and Monetary Policy," Journal of Economic Perspectives, American Economic Association, vol. 11(1), pages 33-49, Winter.
  9. Hansen, Bruce E., 2000. "Testing for structural change in conditional models," Journal of Econometrics, Elsevier, vol. 97(1), pages 93-115, July.
  10. Kenneth D. West, 1994. "Asymptotic Inference About Predictive Ability," Macroeconomics 9410002, EconWPA.
  11. Michael W. McCracken & Todd E. Clark, 2003. "The Predictive Content of the Output Gap for Inflation: Resolving In-Sample and Out-of-Sample Evidence," Computing in Economics and Finance 2003 183, Society for Computational Economics.
  12. Clements,Michael & Hendry,David, 1998. "Forecasting Economic Time Series," Cambridge Books, Cambridge University Press, number 9780521634809.
  13. Donald W.K. Andrews, 1990. "Tests for Parameter Instability and Structural Change with Unknown Change Point," Cowles Foundation Discussion Papers 943, Cowles Foundation for Research in Economics, Yale University.
  14. Hashem Pesaran & Davide Pettenuzzo & Allan Timmermann, 2006. "Forecasting Time Series Subject to Multiple Structural Breaks," Review of Economic Studies, Wiley Blackwell, vol. 73(4), pages 1057-1084, October.
  15. Donald W.K. Andrews & Werner Ploberger, 1992. "Optimal Tests When a Nuisance Parameter Is Present Only Under the Alternative," Cowles Foundation Discussion Papers 1015, Cowles Foundation for Research in Economics, Yale University.
  16. repec:cup:etheor:v:11:y:1995:i:4:p:699-720 is not listed on IDEAS
  17. Andrews, Donald W K, 1991. "Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation," Econometrica, Econometric Society, vol. 59(3), pages 817-58, May.
  18. Andrew J. Patton & Allan Timmermann, 2005. "Testable Implications of Forecast Optimality," STICERD - Econometrics Paper Series /2005/485, Suntory and Toyota International Centres for Economics and Related Disciplines, LSE.
  19. Christoffersen, Peter F. & Diebold, Francis X., 1997. "Optimal Prediction Under Asymmetric Loss," Econometric Theory, Cambridge University Press, vol. 13(06), pages 808-817, December.
  20. Mc Cracken, Michael W., 2000. "Robust out-of-sample inference," Journal of Econometrics, Elsevier, vol. 99(2), pages 195-223, December.
  21. repec:cup:etheor:v:13:y:1997:i:6:p:808-17 is not listed on IDEAS
  22. Chu, Chia-Shang James & Hornik, Kurt & Kuan, Chung-Ming, 1995. "The Moving-Estimates Test for Parameter Stability," Econometric Theory, Cambridge University Press, vol. 11(04), pages 699-720, August.
  23. Jonas D. M. Fisher & Chin Te Liu & Ruilin Zhou, 2002. "When can we forecast inflation?," Economic Perspectives, Federal Reserve Bank of Chicago, issue Q I, pages 32-44.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. Pesaran, M.H. & Pick, A. & Pranovich, M., 2011. "Optimal Forecasts in the Presence of Structural Breaks (Updated 14 November 2011)," Cambridge Working Papers in Economics 1163, Faculty of Economics, University of Cambridge.
  2. Barbara Rossi, 2011. "Advances in Forecasting Under Instability," Working Papers 11-20, Duke University, Department of Economics.
  3. Calhoun, Gray, 2012. "Out-Of-Sample Comparisons of Overfit Models," Staff General Research Papers 32462, Iowa State University, Department of Economics.
  4. Roland Döhrn & Christoph M. Schmidt & Tobias Zimmermann, 2008. "Inflation Forecasting with Inflation Sentiment Indicators," Ruhr Economic Papers 0080, Rheinisch-Westfälisches Institut für Wirtschaftsforschung, Ruhr-Universität Bochum, Universität Dortmund, Universität Duisburg-Essen.
  5. Travis J. Berge, 2011. "Forecasting disconnected exchange rates," Research Working Paper RWP 11-12, Federal Reserve Bank of Kansas City.
  6. David C. Wheelock & Mark E. Wohar, 2009. "Can the term spread predict output growth and recessions? a survey of the literature," Review, Federal Reserve Bank of St. Louis, issue Sep, pages 419-440.
  7. Todd Clark & Michael W. McCracken, 2011. "Advances in forecast evaluation," Working Paper 1120, Federal Reserve Bank of Cleveland.
  8. Prasad S Bhattacharya & Dimitrios D Thomakos, 2011. "Improving forecasting performance by window and model averaging," Economics Series 2011_1, Deakin University, Faculty of Business and Law, School of Accounting, Economics and Finance.
  9. Doyle, Matthew, 2006. "Empirical Phillips Curves in OECD Countries: Has There Been A Common Breakdown?," Staff General Research Papers 12684, Iowa State University, Department of Economics.
  10. Todd E. Clark & Michael W. McCracken, 2008. "Tests of equal predictive ability with real-time data," Working Papers 2008-029, Federal Reserve Bank of St. Louis.
  11. M Hashem Pesaran & Andreas Pick & Mikhail Pranovich, 2011. "Optimal Forecasts in the Presence of Structural Breaks," DNB Working Papers 327, Netherlands Central Bank, Research Department.
  12. Kai Carstensen, 2007. "Is core money growth a good and stable inflation predictor in the euro area?," Kiel Working Papers 1318, Kiel Institute for the World Economy.
  13. David Hendry & Michael P. Clements, 2010. "Forecasting from Mis-specified Models in the Presence of Unanticipated Location Shifts," Economics Series Working Papers 484, University of Oxford, Department of Economics.
  14. Todd E. Clark & Michael W. McCracken, 2010. "Testing for unconditional predictive ability," Working Papers 2010-031, Federal Reserve Bank of St. Louis.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:ecb:ecbwps:20060638

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Official Publications).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.