IDEAS home Printed from https://ideas.repec.org/p/bos/wpaper/wp2015-012.html
   My bibliography  Save this paper

Forecasting in the presence of in and out of sample breaks

Author

Listed:
  • Jiawen Xu

    (Shanghai University of Finance and Economics)

  • Pierre Perron

    (Boston University)

Abstract

We present a frequentist-based approach to forecast time series in the presence of in-sample and out-of-sample breaks in the parameters of the forecasting model. We first model the parameters as following a random level shift process, with the occurrence of a shift governed by a Bernoulli process. In order to have a structure so that changes in the parameters be forecastable, we introduce two modifications. The Örst models the probability of shifts according to some covariates that can be forecasted. The second incorporates a built-in mean reversion mechanism to the time path of the parameters. Similar modifications can also be made to model changes in the variance of the error process. Our full model can be cast into a non-linear nonGaussian state space framework. To estimate it, we use particle filtering and a Monte Carlo expectation maximization algorithm. Simulation results show that the algorithm delivers accurate in-sample estimates, in particular the filtered estimates of the time path of the parameters follow closely their true variations. We provide a number of empirical applications and compare the forecasting performance of our approach with a variety of alternative methods. These show that substantial gains in forecasting accuracy are obtained.

Suggested Citation

  • Jiawen Xu & Pierre Perron, 2015. "Forecasting in the presence of in and out of sample breaks," Boston University - Department of Economics - Working Papers Series wp2015-012, Boston University - Department of Economics.
  • Handle: RePEc:bos:wpaper:wp2015-012
    as

    Download full text from publisher

    File URL: http://people.bu.edu/perron/papers/rls-forecast.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Martin Lettau & Stijn Van Nieuwerburgh, 2008. "Reconciling the Return Predictability Evidence," The Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1607-1652, July.
    2. Ivo Welch & Amit Goyal, 2008. "A Comprehensive Look at The Empirical Performance of Equity Premium Prediction," The Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1455-1508, July.
    3. Jesús Fernández-Villaverde & Juan F. Rubio-Ramírez, 2007. "Estimating Macroeconomic Models: A Likelihood Approach," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 74(4), pages 1059-1087.
    4. Clements, Michael P. & Hendry, David F., 2006. "Forecasting with Breaks," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 12, pages 605-657, Elsevier.
    5. Rossi, Barbara, 2006. "Are Exchange Rates Really Random Walks? Some Evidence Robust To Parameter Instability," Macroeconomic Dynamics, Cambridge University Press, vol. 10(1), pages 20-38, February.
    6. Rong Chen & Jun S. Liu, 2000. "Mixture Kalman filters," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 62(3), pages 493-508.
    7. Godsill, Simon J. & Doucet, Arnaud & West, Mike, 2004. "Monte Carlo Smoothing for Nonlinear Time Series," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 156-168, January.
    8. Lu, Yang K. & Perron, Pierre, 2010. "Modeling and forecasting stock return volatility using a random level shift model," Journal of Empirical Finance, Elsevier, vol. 17(1), pages 138-156, January.
    9. Pettenuzzo, Davide & Timmermann, Allan & Valkanov, Rossen, 2014. "Forecasting stock returns under economic constraints," Journal of Financial Economics, Elsevier, vol. 114(3), pages 517-553.
    10. Sjoerd van den Hauwe & Richard Paap & Dick J.C. van Dijk, 2011. "An Alternative Bayesian Approach to Structural Breaks in Time Series Models," Tinbergen Institute Discussion Papers 11-023/4, Tinbergen Institute.
    11. Ľluboš Pástor & Robert F. Stambaugh, 2001. "The Equity Premium and Structural Breaks," Journal of Finance, American Finance Association, vol. 56(4), pages 1207-1239, August.
    12. Garcia, Rene & Perron, Pierre, 1996. "An Analysis of the Real Interest Rate under Regime Shifts," The Review of Economics and Statistics, MIT Press, vol. 78(1), pages 111-125, February.
    13. Malik, Sheheryar & Pitt, Michael K., 2009. "Modelling Stochastic Volatility with Leverage and Jumps: A Simulated Maximum Likelihood Approach via Particle Filtering," Economic Research Papers 271302, University of Warwick - Department of Economics.
    14. Chib, Siddhartha & Nardari, Federico & Shephard, Neil, 2006. "Analysis of high dimensional multivariate stochastic volatility models," Journal of Econometrics, Elsevier, vol. 134(2), pages 341-371, October.
    15. Kilian, Lutz & Taylor, Mark P., 2003. "Why is it so difficult to beat the random walk forecast of exchange rates?," Journal of International Economics, Elsevier, vol. 60(1), pages 85-107, May.
    16. Chow, Gregory C., 1984. "Random and changing coefficient models," Handbook of Econometrics, in: Z. Griliches† & M. D. Intriligator (ed.), Handbook of Econometrics, edition 1, volume 2, chapter 21, pages 1213-1245, Elsevier.
    17. Pesaran, M. Hashem & Timmermann, Allan, 2002. "Market timing and return prediction under model instability," Journal of Empirical Finance, Elsevier, vol. 9(5), pages 495-510, December.
    18. Sarno,Lucio & Taylor,Mark P., 2003. "The Economics of Exchange Rates," Cambridge Books, Cambridge University Press, number 9780521485845.
    19. Gary Koop & Simon M. Potter, 2007. "Estimation and Forecasting in Models with Multiple Breaks," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 74(3), pages 763-789.
    20. Rasmus T. Varneskov & Pierre Perron, 2018. "Combining long memory and level shifts in modelling and forecasting the volatility of asset returns," Quantitative Finance, Taylor & Francis Journals, vol. 18(3), pages 371-393, March.
    21. Giordani, Paolo & Kohn, Robert, 2008. "Efficient Bayesian Inference for Multiple Change-Point and Mixture Innovation Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 26, pages 66-77, January.
    22. Peter R. Hansen & Asger Lunde & James M. Nason, 2011. "The Model Confidence Set," Econometrica, Econometric Society, vol. 79(2), pages 453-497, March.
    23. Jan J. J. Groen & Richard Paap & Francesco Ravazzolo, 2013. "Real-Time Inflation Forecasting in a Changing World," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 31(1), pages 29-44, January.
    24. Giordani, Paolo & Kohn, Robert & van Dijk, Dick, 2007. "A unified approach to nonlinearity, structural change, and outliers," Journal of Econometrics, Elsevier, vol. 137(1), pages 112-133, March.
    25. Jushan Bai & Pierre Perron, 1998. "Estimating and Testing Linear Models with Multiple Structural Changes," Econometrica, Econometric Society, vol. 66(1), pages 47-78, January.
    26. Yihong Xia, 2001. "Learning about Predictability: The Effects of Parameter Uncertainty on Dynamic Asset Allocation," Journal of Finance, American Finance Association, vol. 56(1), pages 205-246, February.
    27. Perron, Pierre & Qu, Zhongjun, 2010. "Long-Memory and Level Shifts in the Volatility of Stock Market Return Indices," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(2), pages 275-290.
    28. M. Hashem Pesaran & Davide Pettenuzzo & Allan Timmermann, 2006. "Forecasting Time Series Subject to Multiple Structural Breaks," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 73(4), pages 1057-1084.
    29. Sangjoon Kim & Neil Shephard & Siddhartha Chib, 1998. "Stochastic Volatility: Likelihood Inference and Comparison with ARCH Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 65(3), pages 361-393.
    30. Andrews, Donald W K, 1993. "Tests for Parameter Instability and Structural Change with Unknown Change Point," Econometrica, Econometric Society, vol. 61(4), pages 821-856, July.
    31. Xu, Jiawen & Perron, Pierre, 2014. "Forecasting return volatility: Level shifts with varying jump probability and mean reversion," International Journal of Forecasting, Elsevier, vol. 30(3), pages 449-463.
    32. Olsson, Jimmy & Rydén, Tobias, 2008. "Asymptotic properties of particle filter-based maximum likelihood estimators for state space models," Stochastic Processes and their Applications, Elsevier, vol. 118(4), pages 649-680, April.
    33. Pettenuzzo, Davide & Timmermann, Allan, 2011. "Predictability of stock returns and asset allocation under structural breaks," Journal of Econometrics, Elsevier, vol. 164(1), pages 60-78, September.
    34. Blasques, Francisco & Koopman, Siem Jan & Łasak, Katarzyna & Lucas, André, 2016. "In-sample confidence bands and out-of-sample forecast bands for time-varying parameters in observation-driven models," International Journal of Forecasting, Elsevier, vol. 32(3), pages 875-887.
    35. John M. Maheu & Stephen Gordon, 2008. "Learning, forecasting and structural breaks," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 23(5), pages 553-583.
    36. Ang, Andrew & Bekaert, Geert, 2002. "Regime Switches in Interest Rates," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 163-182, April.
    37. Meese, Richard A. & Rogoff, Kenneth, 1983. "Empirical exchange rate models of the seventies : Do they fit out of sample?," Journal of International Economics, Elsevier, vol. 14(1-2), pages 3-24, February.
    38. Maheu, John M. & McCurdy, Thomas H., 2009. "How Useful are Historical Data for Forecasting the Long-Run Equity Return Distribution?," Journal of Business & Economic Statistics, American Statistical Association, vol. 27, pages 95-112.
    39. Malik, Sheheryar & Pitt, Michael K, 2009. "Modelling Stochastic Volatility with Leverage and Jumps : A Simulated Maximum Likelihood Approach via Particle Filtering," The Warwick Economics Research Paper Series (TWERPS) 897, University of Warwick, Department of Economics.
    40. Malik, S. & Pitt, M. K., 2011. "Modelling Stochastic Volatility with Leverage and Jumps: A Simulated Maximum Likelihood Approach via Particle Filtering," Working papers 318, Banque de France.
    41. Ravi Jagannathan & Ellen R. McGrattan & Anna Scherbina, 2000. "The declining U.S. equity premium," Quarterly Review, Federal Reserve Bank of Minneapolis, vol. 24(Fall), pages 3-19.
    42. Zhongjun Qu & Pierre Perron, 2013. "A stochastic volatility model with random level shifts and its applications to S&P 500 and NASDAQ return indices," Econometrics Journal, Royal Economic Society, vol. 16(3), pages 309-339, October.
    43. Paye, Bradley S. & Timmermann, Allan, 2006. "Instability of return prediction models," Journal of Empirical Finance, Elsevier, vol. 13(3), pages 274-315, June.
    44. Barr Rosenberg, 1973. "The Analysis of a Cross Section of Time Series by Stochastically Convergent Parameter Regression," NBER Chapters, in: Annals of Economic and Social Measurement, Volume 2, number 4, pages 399-428, National Bureau of Economic Research, Inc.
    45. Bonnie K. Ray & Ruey S. Tsay, 2002. "Bayesian methods for change‐point detection in long‐range dependent processes," Journal of Time Series Analysis, Wiley Blackwell, vol. 23(6), pages 687-705, November.
    46. Fama, Eugene F. & French, Kenneth R., 1993. "Common risk factors in the returns on stocks and bonds," Journal of Financial Economics, Elsevier, vol. 33(1), pages 3-56, February.
    47. David E. Rapach & Mark E. Wohar, 2006. "Structural Breaks and Predictive Regression Models of Aggregate U.S. Stock Returns," Journal of Financial Econometrics, Oxford University Press, vol. 4(2), pages 238-274.
    48. James H. Stock & Mark W. Watson, 2007. "Why Has U.S. Inflation Become Harder to Forecast?," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 39(s1), pages 3-33, February.
    49. Stock, James H & Watson, Mark W, 1996. "Evidence on Structural Instability in Macroeconomic Time Series Relations," Journal of Business & Economic Statistics, American Statistical Association, vol. 14(1), pages 11-30, January.
    50. Harvey, Andrew, 2006. "Forecasting with Unobserved Components Time Series Models," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 7, pages 327-412, Elsevier.
    51. Ye Li & Pierre Perron & Jiawen Xu, 2017. "Modelling exchange rate volatility with random level shifts," Applied Economics, Taylor & Francis Journals, vol. 49(26), pages 2579-2589, June.
    52. Smith Aaron, 2012. "Markov Breaks in Regression Models," Journal of Time Series Econometrics, De Gruyter, vol. 4(1), pages 1-35, May.
    53. Chib, Siddhartha, 1998. "Estimation and comparison of multiple change-point models," Journal of Econometrics, Elsevier, vol. 86(2), pages 221-241, June.
    54. Pitt, Michael K, 2002. "Smooth Particle Filters for Likelihood Evaluation and Maximisation," The Warwick Economics Research Paper Series (TWERPS) 651, University of Warwick, Department of Economics.
    55. Molodtsova, Tanya & Papell, David H., 2009. "Out-of-sample exchange rate predictability with Taylor rule fundamentals," Journal of International Economics, Elsevier, vol. 77(2), pages 167-180, April.
    56. Jushan Bai & Pierre Perron, 2003. "Computation and analysis of multiple structural change models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 18(1), pages 1-22.
    57. Hull, John C & White, Alan D, 1987. "The Pricing of Options on Assets with Stochastic Volatilities," Journal of Finance, American Finance Association, vol. 42(2), pages 281-300, June.
    58. Taylor, John B., 1993. "Discretion versus policy rules in practice," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 39(1), pages 195-214, December.
    59. Andrew Atkeson & Lee E. Ohanian, 2001. "Are Phillips curves useful for forecasting inflation?," Quarterly Review, Federal Reserve Bank of Minneapolis, vol. 25(Win), pages 2-11.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Pierre Perron & Yohei Yamamoto, 2021. "Testing for Changes in Forecasting Performance," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(1), pages 148-165, January.
    2. Ye Li & Pierre Perron & Jiawen Xu, 2017. "Modelling exchange rate volatility with random level shifts," Applied Economics, Taylor & Francis Journals, vol. 49(26), pages 2579-2589, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jiawen Xu & Pierre Perron, 2023. "Forecasting in the presence of in-sample and out-of-sample breaks," Empirical Economics, Springer, vol. 64(6), pages 3001-3035, June.
    2. Jiawen Xu & Pierre Perron, 2015. "Forecasting in the presence of in and out of sample breaks," Boston University - Department of Economics - Working Papers Series wp2015-012, Boston University - Department of Economics.
    3. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    4. Geweke, John & Jiang, Yu, 2011. "Inference and prediction in a multiple-structural-break model," Journal of Econometrics, Elsevier, vol. 163(2), pages 172-185, August.
    5. Sjoerd van den Hauwe & Richard Paap & Dick J.C. van Dijk, 2011. "An Alternative Bayesian Approach to Structural Breaks in Time Series Models," Tinbergen Institute Discussion Papers 11-023/4, Tinbergen Institute.
    6. Nima Nonejad, 2021. "An Overview Of Dynamic Model Averaging Techniques In Time‐Series Econometrics," Journal of Economic Surveys, Wiley Blackwell, vol. 35(2), pages 566-614, April.
    7. Narayan, Seema & Smyth, Russell, 2015. "The financial econometrics of price discovery and predictability," International Review of Financial Analysis, Elsevier, vol. 42(C), pages 380-393.
    8. Simon C. Smith, 2020. "Equity premium prediction and structural breaks," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 25(3), pages 412-429, July.
    9. Maheu, John M. & McCurdy, Thomas H., 2009. "How Useful are Historical Data for Forecasting the Long-Run Equity Return Distribution?," Journal of Business & Economic Statistics, American Statistical Association, vol. 27, pages 95-112.
    10. Smith, Simon C., 2017. "Equity premium estimates from economic fundamentals under structural breaks," International Review of Financial Analysis, Elsevier, vol. 52(C), pages 49-61.
    11. Boot, Tom & Pick, Andreas, 2020. "Does modeling a structural break improve forecast accuracy?," Journal of Econometrics, Elsevier, vol. 215(1), pages 35-59.
    12. Rapach, David & Zhou, Guofu, 2013. "Forecasting Stock Returns," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 328-383, Elsevier.
    13. Allan Timmermann, 2018. "Forecasting Methods in Finance," Annual Review of Financial Economics, Annual Reviews, vol. 10(1), pages 449-479, November.
    14. Dufays, Arnaud & Rombouts, Jeroen V.K., 2020. "Relevant parameter changes in structural break models," Journal of Econometrics, Elsevier, vol. 217(1), pages 46-78.
    15. Ardia, David & Dufays, Arnaud & Ordás Criado, Carlos, 2023. "Linking Frequentist and Bayesian Change-Point Methods," MPRA Paper 119486, University Library of Munich, Germany.
    16. M. Hashem Pesaran & Davide Pettenuzzo & Allan Timmermann, 2006. "Forecasting Time Series Subject to Multiple Structural Breaks," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 73(4), pages 1057-1084.
    17. Gary Koop & Simon M. Potter, 2009. "Prior Elicitation In Multiple Change-Point Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 50(3), pages 751-772, August.
    18. Dangl, Thomas & Halling, Michael, 2012. "Predictive regressions with time-varying coefficients," Journal of Financial Economics, Elsevier, vol. 106(1), pages 157-181.
    19. Michael L. Polemis & Thanasis Stengos, 2019. "Does competition prevent industrial pollution? Evidence from a panel threshold model," Business Strategy and the Environment, Wiley Blackwell, vol. 28(1), pages 98-110, January.
    20. Andrew Ang & Allan Timmermann, 2012. "Regime Changes and Financial Markets," Annual Review of Financial Economics, Annual Reviews, vol. 4(1), pages 313-337, October.

    More about this item

    Keywords

    instabilities; structural change; forecasting; random level shifts; particle filter;
    All these keywords.

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bos:wpaper:wp2015-012. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Program Coordinator (email available below). General contact details of provider: https://edirc.repec.org/data/decbuus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.