IDEAS home Printed from https://ideas.repec.org/a/eee/econom/v215y2020i1p35-59.html
   My bibliography  Save this article

Does modeling a structural break improve forecast accuracy?

Author

Listed:
  • Boot, Tom
  • Pick, Andreas

Abstract

Mean square forecast error loss implies a bias–variance trade-off that suggests that structural breaks of small magnitude should be ignored. In this paper, we provide a test to determine whether modeling a structural break improves forecast accuracy. The test is near optimal even when the date of a local-to-zero break is not consistently estimable. The results extend to forecast combinations that weight the post-break sample and the full sample forecasts by our test statistic. In a large number of macroeconomic time series, we find that structural breaks that are relevant for forecasting occur much less frequently than existing tests indicate.

Suggested Citation

  • Boot, Tom & Pick, Andreas, 2020. "Does modeling a structural break improve forecast accuracy?," Journal of Econometrics, Elsevier, vol. 215(1), pages 35-59.
    • Handle: RePEc:eee:econom:v:215:y:2020:i:1:p:35-59
    DOI: 10.1016/j.jeconom.2019.07.007
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304407619301824
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jeconom.2019.07.007?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Pesaran, M. Hashem & Timmermann, Allan, 2005. "Small sample properties of forecasts from autoregressive models under structural breaks," Journal of Econometrics, Elsevier, vol. 129(1-2), pages 183-217.
    2. Rossi, Barbara, 2006. "Are Exchange Rates Really Random Walks? Some Evidence Robust To Parameter Instability," Macroeconomic Dynamics, Cambridge University Press, vol. 10(1), pages 20-38, February.
    3. Graham Elliott & Ulrich K. Müller & Mark W. Watson, 2015. "Nearly Optimal Tests When a Nuisance Parameter Is Present Under the Null Hypothesis," Econometrica, Econometric Society, vol. 83, pages 771-811, March.
    4. Inoue, Atsushi & Jin, Lu & Rossi, Barbara, 2017. "Rolling window selection for out-of-sample forecasting with time-varying parameters," Journal of Econometrics, Elsevier, vol. 196(1), pages 55-67.
    5. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
    6. Raffaella Giacomini & Barbara Rossi, 2009. "Detecting and Predicting Forecast Breakdowns," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(2), pages 669-705.
    7. Michael W. McCracken & Serena Ng, 2016. "FRED-MD: A Monthly Database for Macroeconomic Research," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(4), pages 574-589, October.
    8. Elliott, Graham & Muller, Ulrich K., 2007. "Confidence sets for the date of a single break in linear time series regressions," Journal of Econometrics, Elsevier, vol. 141(2), pages 1196-1218, December.
    9. Clark, Todd E. & McCracken, Michael W., 2012. "In-sample tests of predictive ability: A new approach," Journal of Econometrics, Elsevier, vol. 170(1), pages 1-14.
    10. Ľluboš Pástor & Robert F. Stambaugh, 2001. "The Equity Premium and Structural Breaks," Journal of Finance, American Finance Association, vol. 56(4), pages 1207-1239, August.
    11. Pesaran, M. Hashem & Pick, Andreas & Pranovich, Mikhail, 2013. "Optimal forecasts in the presence of structural breaks," Journal of Econometrics, Elsevier, vol. 177(2), pages 134-152.
    12. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    13. Pesaran, M. Hashem & Timmermann, Allan, 2002. "Market timing and return prediction under model instability," Journal of Empirical Finance, Elsevier, vol. 9(5), pages 495-510, December.
    14. Hausman, Jerry, 2015. "Specification tests in econometrics," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 38(2), pages 112-134.
    15. Dufour, Jean-Marie & Ghysels, Eric & Hall, Alastair, 1994. "Generalized Predictive Tests and Structural Change Analysis in Econometrics," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 35(1), pages 199-229, February.
    16. Elliott, Graham & Müller, Ulrich K., 2014. "Pre and post break parameter inference," Journal of Econometrics, Elsevier, vol. 180(2), pages 141-157.
    17. Jushan Bai & Pierre Perron, 1998. "Estimating and Testing Linear Models with Multiple Structural Changes," Econometrica, Econometric Society, vol. 66(1), pages 47-78, January.
    18. Phillips, Peter C. B., 1979. "The sampling distribution of forecasts from a first-order autoregression," Journal of Econometrics, Elsevier, vol. 9(3), pages 241-261, February.
    19. Clark, Todd & McCracken, Michael, 2013. "Advances in Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1107-1201, Elsevier.
    20. Siddhartha Chib & Kyu Ho Kang, 2013. "Change-Points in Affine Arbitrage-Free Term Structure Models," Journal of Financial Econometrics, Oxford University Press, vol. 11(2), pages 302-334, March.
    21. Andrews, Donald W K, 1993. "Tests for Parameter Instability and Structural Change with Unknown Change Point," Econometrica, Econometric Society, vol. 61(4), pages 821-856, July.
    22. Pettenuzzo, Davide & Timmermann, Allan, 2011. "Predictability of stock returns and asset allocation under structural breaks," Journal of Econometrics, Elsevier, vol. 164(1), pages 60-78, September.
    23. Pesaran, M. Hashem & Pick, Andreas & Timmermann, Allan, 2011. "Variable selection, estimation and inference for multi-period forecasting problems," Journal of Econometrics, Elsevier, vol. 164(1), pages 173-187, September.
    24. Holger Dette & Dominik Wied, 2016. "Detecting relevant changes in time series models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(2), pages 371-394, March.
    25. Paye, Bradley S. & Timmermann, Allan, 2006. "Instability of return prediction models," Journal of Empirical Finance, Elsevier, vol. 13(3), pages 274-315, June.
    26. Andrews, Donald W K & Ploberger, Werner, 1994. "Optimal Tests When a Nuisance Parameter Is Present Only under the Alternative," Econometrica, Econometric Society, vol. 62(6), pages 1383-1414, November.
    27. Raffaella Giacomini & Barbara Rossi, 2010. "Forecast comparisons in unstable environments," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 595-620.
    28. David E. Rapach & Mark E. Wohar, 2006. "Structural Breaks and Predictive Regression Models of Aggregate U.S. Stock Returns," Journal of Financial Econometrics, Oxford University Press, vol. 4(2), pages 238-274.
    29. James H. Stock & Mark W. Watson, 2007. "Why Has U.S. Inflation Become Harder to Forecast?," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 39(s1), pages 3-33, February.
    30. Hansen, Bruce E., 2009. "Averaging Estimators For Regressions With A Possible Structural Break," Econometric Theory, Cambridge University Press, vol. 25(6), pages 1498-1514, December.
    31. Stock, James H & Watson, Mark W, 1996. "Evidence on Structural Instability in Macroeconomic Time Series Relations," Journal of Business & Economic Statistics, American Statistical Association, vol. 14(1), pages 11-30, January.
    32. MacKinnon, James G. & White, Halbert, 1985. "Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties," Journal of Econometrics, Elsevier, vol. 29(3), pages 305-325, September.
    33. Pesaran, M. Hashem & Timmermann, Allan, 2007. "Selection of estimation window in the presence of breaks," Journal of Econometrics, Elsevier, vol. 137(1), pages 134-161, March.
    34. Wallace, T D, 1972. "Weaker Criteria and Tests for Linear Restrictions in Regression," Econometrica, Econometric Society, vol. 40(4), pages 689-698, July.
    35. Ploberger, Werner & Kramer, Walter & Kontrus, Karl, 1989. "A new test for structural stability in the linear regression model," Journal of Econometrics, Elsevier, vol. 40(2), pages 307-318, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Tae‐Hwy Lee & Shahnaz Parsaeian & Aman Ullah, 2022. "Forecasting Under Structural Breaks Using Improved Weighted Estimation," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 84(6), pages 1485-1501, December.
    2. Philippe Goulet Coulombe & Maxime Leroux & Dalibor Stevanovic & Stéphane Surprenant, 2022. "How is machine learning useful for macroeconomic forecasting?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 920-964, August.
    3. Ballinari, Daniele & Behrendt, Simon, 2020. "Structural breaks in online investor sentiment: A note on the nonstationarity of financial chatter," Finance Research Letters, Elsevier, vol. 35(C).
    4. Karsten Schweikert, 2022. "Oracle Efficient Estimation of Structural Breaks in Cointegrating Regressions," Journal of Time Series Analysis, Wiley Blackwell, vol. 43(1), pages 83-104, January.
    5. Tae‐Hwy Lee & Shahnaz Parsaeian & Aman Ullah, 2022. "Optimal forecast under structural breaks," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 965-987, August.
    6. Timmermann, Allan & Pesaran, M. Hashem & Pick, Andreas, 2022. "Forecasting with panel data: estimation uncertainty versus parameter heterogeneity," CEPR Discussion Papers 17123, C.E.P.R. Discussion Papers.
    7. Yu Jeffrey Hu & Jeroen Rombouts & Ines Wilms, 2023. "Fast Forecasting of Unstable Data Streams for On-Demand Service Platforms," Papers 2303.01887, arXiv.org.
    8. Gantungalag Altansukh & Denise R. Osborn, 2022. "Using structural break inference for forecasting time series," Empirical Economics, Springer, vol. 63(1), pages 1-41, July.
    9. Renata Tavanielli & Márcio Laurini, 2023. "Yield Curve Models with Regime Changes: An Analysis for the Brazilian Interest Rate Market," Mathematics, MDPI, vol. 11(11), pages 1-28, June.
    10. Biolsi, Christopher, 2021. "Labor productivity forecasts based on a Beveridge–Nelson filter: Is there statistical evidence for a slowdown?," Journal of Macroeconomics, Elsevier, vol. 69(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tom Boot & Andreas Pick, 2017. "A near optimal test for structural breaks when forecasting under square error loss," Tinbergen Institute Discussion Papers 17-039/III, Tinbergen Institute.
    2. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    3. Yin, Anwen, 2015. "Forecasting and model averaging with structural breaks," ISU General Staff Papers 201501010800005727, Iowa State University, Department of Economics.
    4. Raffaella Giacomini & Barbara Rossi, 2015. "Forecasting in Nonstationary Environments: What Works and What Doesn't in Reduced-Form and Structural Models," Annual Review of Economics, Annual Reviews, vol. 7(1), pages 207-229, August.
    5. Jiawen Xu & Pierre Perron, 2015. "Forecasting in the presence of in and out of sample breaks," Boston University - Department of Economics - Working Papers Series wp2015-012, Boston University - Department of Economics.
    6. Rapach, David & Zhou, Guofu, 2013. "Forecasting Stock Returns," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 328-383, Elsevier.
    7. Raffaella Giacomini & Barbara Rossi, 2013. "Forecasting in macroeconomics," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 17, pages 381-408, Edward Elgar Publishing.
    8. Jiawen Xu & Pierre Perron, 2023. "Forecasting in the presence of in-sample and out-of-sample breaks," Empirical Economics, Springer, vol. 64(6), pages 3001-3035, June.
    9. Tae‐Hwy Lee & Shahnaz Parsaeian & Aman Ullah, 2022. "Optimal forecast under structural breaks," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 965-987, August.
    10. Narayan, Seema & Smyth, Russell, 2015. "The financial econometrics of price discovery and predictability," International Review of Financial Analysis, Elsevier, vol. 42(C), pages 380-393.
    11. Inoue, Atsushi & Jin, Lu & Rossi, Barbara, 2017. "Rolling window selection for out-of-sample forecasting with time-varying parameters," Journal of Econometrics, Elsevier, vol. 196(1), pages 55-67.
    12. Barbara Rossi, 2019. "Forecasting in the presence of instabilities: How do we know whether models predict well and how to improve them," Economics Working Papers 1711, Department of Economics and Business, Universitat Pompeu Fabra, revised Jul 2021.
    13. Gantungalag Altansukh & Denise R. Osborn, 2022. "Using structural break inference for forecasting time series," Empirical Economics, Springer, vol. 63(1), pages 1-41, July.
    14. Yin, Anwen, 2019. "Out-of-sample equity premium prediction in the presence of structural breaks," International Review of Financial Analysis, Elsevier, vol. 65(C).
    15. Odendahl, Florens & Rossi, Barbara & Sekhposyan, Tatevik, 2023. "Evaluating forecast performance with state dependence," Journal of Econometrics, Elsevier, vol. 237(2).
    16. Daniel Borup & Jonas N. Eriksen & Mads M. Kjær & Martin Thyrsgaard, 2020. "Predicting bond return predictability," CREATES Research Papers 2020-09, Department of Economics and Business Economics, Aarhus University.
    17. Jiawen Xu & Pierre Perron, 2015. "Forecasting in the presence of in and out of sample breaks," Boston University - Department of Economics - Working Papers Series wp2015-012, Boston University - Department of Economics.
    18. Chevillon, Guillaume, 2016. "Multistep forecasting in the presence of location shifts," International Journal of Forecasting, Elsevier, vol. 32(1), pages 121-137.
    19. Graham Elliott & Allan Timmermann, 2016. "Forecasting in Economics and Finance," Annual Review of Economics, Annual Reviews, vol. 8(1), pages 81-110, October.
    20. Timmermann, Allan, 2018. "Forecasting Methods in Finance," CEPR Discussion Papers 12692, C.E.P.R. Discussion Papers.

    More about this item

    Keywords

    Structural break test; Forecasting; Squared error loss;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:215:y:2020:i:1:p:35-59. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jeconom .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.