IDEAS home Printed from https://ideas.repec.org/p/tin/wpaper/20170039.html
   My bibliography  Save this paper

A near optimal test for structural breaks when forecasting under square error loss

Author

Listed:
  • Tom Boot

    (University of Groningen)

  • Andreas Pick

    (Erasmus University Rotterdam, De Nederlandsche Bank and CESifo Institute)

Abstract

We propose a near optimal test for structural breaks of unknown timing when the purpose of the analysis is to obtain accurate forecasts under square error loss. A bias-variance trade-off exists under square forecast error loss, which implies that small structural breaks should be ignored. We study critical break sizes, assess the relevance of the break location, and provide a test to determine whether modeling a break will improve forecast accuracy. Asymptotic critical values and near optimality properties are established allowing for a break under the null, where the critical break size varies with the break location. The results are extended to a class of shrinkage forecasts with our test statistic as shrinkage constant. Empirical results on a large number of macroeconomic time series show that structural breaks that are relevant for forecasting occur much less frequently than indicated by existing tests.

Suggested Citation

  • Tom Boot & Andreas Pick, 2017. "A near optimal test for structural breaks when forecasting under square error loss," Tinbergen Institute Discussion Papers 17-039/III, Tinbergen Institute.
  • Handle: RePEc:tin:wpaper:20170039
    as

    Download full text from publisher

    File URL: https://papers.tinbergen.nl/17039.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Pesaran, M. Hashem & Timmermann, Allan, 2005. "Small sample properties of forecasts from autoregressive models under structural breaks," Journal of Econometrics, Elsevier, vol. 129(1-2), pages 183-217.
    2. Rossi, Barbara, 2006. "Are Exchange Rates Really Random Walks? Some Evidence Robust To Parameter Instability," Macroeconomic Dynamics, Cambridge University Press, vol. 10(1), pages 20-38, February.
    3. Graham Elliott & Ulrich K. Müller & Mark W. Watson, 2015. "Nearly Optimal Tests When a Nuisance Parameter Is Present Under the Null Hypothesis," Econometrica, Econometric Society, vol. 83, pages 771-811, March.
    4. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
    5. Raffaella Giacomini & Barbara Rossi, 2009. "Detecting and Predicting Forecast Breakdowns," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(2), pages 669-705.
    6. Michael W. McCracken & Serena Ng, 2016. "FRED-MD: A Monthly Database for Macroeconomic Research," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(4), pages 574-589, October.
    7. Elliott, Graham & Muller, Ulrich K., 2007. "Confidence sets for the date of a single break in linear time series regressions," Journal of Econometrics, Elsevier, vol. 141(2), pages 1196-1218, December.
    8. Danilov, D.L. & Magnus, J.R., 2002. "Estimation of the Mean of a Univariate Normal Distribution When the Variance is not Known," Other publications TiSEM 002a672b-73b6-4a8b-8901-7, Tilburg University, School of Economics and Management.
    9. Clark, Todd E. & McCracken, Michael W., 2012. "In-sample tests of predictive ability: A new approach," Journal of Econometrics, Elsevier, vol. 170(1), pages 1-14.
    10. Pesaran, M. Hashem & Pick, Andreas & Pranovich, Mikhail, 2013. "Optimal forecasts in the presence of structural breaks," Journal of Econometrics, Elsevier, vol. 177(2), pages 134-152.
    11. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    12. Hausman, Jerry, 2015. "Specification tests in econometrics," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 38(2), pages 112-134.
    13. Elliott, Graham & Müller, Ulrich K., 2014. "Pre and post break parameter inference," Journal of Econometrics, Elsevier, vol. 180(2), pages 141-157.
    14. Jushan Bai & Pierre Perron, 1998. "Estimating and Testing Linear Models with Multiple Structural Changes," Econometrica, Econometric Society, vol. 66(1), pages 47-78, January.
    15. Andrews, Donald W K, 1993. "Tests for Parameter Instability and Structural Change with Unknown Change Point," Econometrica, Econometric Society, vol. 61(4), pages 821-856, July.
    16. Holger Dette & Dominik Wied, 2016. "Detecting relevant changes in time series models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(2), pages 371-394, March.
    17. Paye, Bradley S. & Timmermann, Allan, 2006. "Instability of return prediction models," Journal of Empirical Finance, Elsevier, vol. 13(3), pages 274-315, June.
    18. Andrews, Donald W K & Ploberger, Werner, 1994. "Optimal Tests When a Nuisance Parameter Is Present Only under the Alternative," Econometrica, Econometric Society, vol. 62(6), pages 1383-1414, November.
    19. David E. Rapach & Mark E. Wohar, 2006. "Structural Breaks and Predictive Regression Models of Aggregate U.S. Stock Returns," Journal of Financial Econometrics, Oxford University Press, vol. 4(2), pages 238-274.
    20. Hansen, Bruce E., 2009. "Averaging Estimators For Regressions With A Possible Structural Break," Econometric Theory, Cambridge University Press, vol. 25(6), pages 1498-1514, December.
    21. Stock, James H & Watson, Mark W, 1996. "Evidence on Structural Instability in Macroeconomic Time Series Relations," Journal of Business & Economic Statistics, American Statistical Association, vol. 14(1), pages 11-30, January.
    22. MacKinnon, James G. & White, Halbert, 1985. "Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties," Journal of Econometrics, Elsevier, vol. 29(3), pages 305-325, September.
    23. Pesaran, M. Hashem & Timmermann, Allan, 2007. "Selection of estimation window in the presence of breaks," Journal of Econometrics, Elsevier, vol. 137(1), pages 134-161, March.
    24. Jan R. Magnus, 2002. "Estimation of the mean of a univariate normal distribution with known variance," Econometrics Journal, Royal Economic Society, vol. 5(1), pages 225-236, June.
    25. Wallace, T D, 1972. "Weaker Criteria and Tests for Linear Restrictions in Regression," Econometrica, Econometric Society, vol. 40(4), pages 689-698, July.
    26. Ploberger, Werner & Kramer, Walter & Kontrus, Karl, 1989. "A new test for structural stability in the linear regression model," Journal of Econometrics, Elsevier, vol. 40(2), pages 307-318, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Boot, Tom & Pick, Andreas, 2020. "Does modeling a structural break improve forecast accuracy?," Journal of Econometrics, Elsevier, vol. 215(1), pages 35-59.
    2. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    3. Yin, Anwen, 2015. "Forecasting and model averaging with structural breaks," ISU General Staff Papers 201501010800005727, Iowa State University, Department of Economics.
    4. Raffaella Giacomini & Barbara Rossi, 2015. "Forecasting in Nonstationary Environments: What Works and What Doesn't in Reduced-Form and Structural Models," Annual Review of Economics, Annual Reviews, vol. 7(1), pages 207-229, August.
    5. Barbara Rossi, 2019. "Forecasting in the presence of instabilities: How do we know whether models predict well and how to improve them," Economics Working Papers 1711, Department of Economics and Business, Universitat Pompeu Fabra, revised Jul 2021.
    6. Inoue, Atsushi & Jin, Lu & Rossi, Barbara, 2017. "Rolling window selection for out-of-sample forecasting with time-varying parameters," Journal of Econometrics, Elsevier, vol. 196(1), pages 55-67.
    7. Gantungalag Altansukh & Denise R. Osborn, 2022. "Using structural break inference for forecasting time series," Empirical Economics, Springer, vol. 63(1), pages 1-41, July.
    8. Yin, Anwen, 2019. "Out-of-sample equity premium prediction in the presence of structural breaks," International Review of Financial Analysis, Elsevier, vol. 65(C).
    9. Hännikäinen Jari, 2017. "Selection of an Estimation Window in the Presence of Data Revisions and Recent Structural Breaks," Journal of Econometric Methods, De Gruyter, vol. 6(1), pages 1-22, January.
    10. Odendahl, Florens & Rossi, Barbara & Sekhposyan, Tatevik, 2023. "Evaluating forecast performance with state dependence," Journal of Econometrics, Elsevier, vol. 237(2).
    11. Mwasi Paza Mboya & Philipp Sibbertsen, 2023. "Optimal forecasts in the presence of discrete structural breaks under long memory," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(7), pages 1889-1908, November.
    12. Tae‐Hwy Lee & Shahnaz Parsaeian & Aman Ullah, 2022. "Optimal forecast under structural breaks," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 965-987, August.
    13. Graham Elliott & Allan Timmermann, 2016. "Forecasting in Economics and Finance," Annual Review of Economics, Annual Reviews, vol. 8(1), pages 81-110, October.
    14. Zongwu Cai & Seong Yeon Chang, 2018. "A New Test In A Predictive Regression with Structural Breaks," WORKING PAPERS SERIES IN THEORETICAL AND APPLIED ECONOMICS 201811, University of Kansas, Department of Economics, revised Dec 2018.
    15. Rossi, Barbara & Inoue, Atsushi & Jin, Lu, 2014. "Window Selection for Out-of-Sample Forecasting with Time-Varying Parameters," CEPR Discussion Papers 10168, C.E.P.R. Discussion Papers.
    16. Narayan, Seema & Smyth, Russell, 2015. "The financial econometrics of price discovery and predictability," International Review of Financial Analysis, Elsevier, vol. 42(C), pages 380-393.
    17. Rapach, David & Zhou, Guofu, 2013. "Forecasting Stock Returns," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 328-383, Elsevier.
    18. Koo, Bonsoo & Seo, Myung Hwan, 2015. "Structural-break models under mis-specification: Implications for forecasting," Journal of Econometrics, Elsevier, vol. 188(1), pages 166-181.
    19. Geweke, J. & Joel Horowitz & Pesaran, M.H., 2006. "Econometrics: A Bird’s Eye View," Cambridge Working Papers in Economics 0655, Faculty of Economics, University of Cambridge.
    20. Andersen, Torben G. & Varneskov, Rasmus T., 2022. "Testing for parameter instability and structural change in persistent predictive regressions," Journal of Econometrics, Elsevier, vol. 231(2), pages 361-386.

    More about this item

    Keywords

    structural break test; forecasting; squared error loss;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tin:wpaper:20170039. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Tinbergen Office +31 (0)10-4088900 (email available below). General contact details of provider: https://edirc.repec.org/data/tinbenl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.