IDEAS home Printed from https://ideas.repec.org/p/ehl/lserod/103430.html
   My bibliography  Save this paper

Detecting possibly frequent change-points: Wild Binary Segmentation 2 and steepest-drop model selection

Author

Listed:
  • Fryzlewicz, Piotr

Abstract

Many existing procedures for detecting multiple change-points in data sequences fail in frequent-change-point scenarios. This article proposes a new change-point detection methodology designed to work well in both infrequent and frequent change-point settings. It is made up of two ingredients: one is “Wild Binary Segmentation 2” (WBS2), a recursive algorithm for producing what we call a ‘complete’ solution path to the change-point detection problem, i.e. a sequence of estimated nested models containing 0 , … , T- 1 change-points, where T is the data length. The other ingredient is a new model selection procedure, referred to as “Steepest Drop to Low Levels” (SDLL). The SDLL criterion acts on the WBS2 solution path, and, unlike many existing model selection procedures for change-point problems, it is not penalty-based, and only uses thresholding as a certain discrete secondary check. The resulting WBS2.SDLL procedure, combining both ingredients, is shown to be consistent, and to significantly outperform the competition in the frequent change-point scenarios tested. WBS2.SDLL is fast, easy to code and does not require the choice of a window or span parameter.

Suggested Citation

  • Fryzlewicz, Piotr, 2020. "Detecting possibly frequent change-points: Wild Binary Segmentation 2 and steepest-drop model selection," LSE Research Online Documents on Economics 103430, London School of Economics and Political Science, LSE Library.
  • Handle: RePEc:ehl:lserod:103430
    as

    Download full text from publisher

    File URL: http://eprints.lse.ac.uk/103430/
    File Function: Open access version.
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Zeileis, Achim & Leisch, Friedrich & Hornik, Kurt & Kleiber, Christian, 2002. "strucchange: An R Package for Testing for Structural Change in Linear Regression Models," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 7(i02).
    2. Harchaoui, Z. & Lévy-Leduc, C., 2010. "Multiple Change-Point Estimation With a Total Variation Penalty," Journal of the American Statistical Association, American Statistical Association, vol. 105(492), pages 1480-1493.
    3. Marc Lavielle & Eric Moulines, 2000. "Least‐squares Estimation of an Unknown Number of Shifts in a Time Series," Journal of Time Series Analysis, Wiley Blackwell, vol. 21(1), pages 33-59, January.
    4. Gabriela Ciuperca, 2014. "Model selection by LASSO methods in a change-point model," Statistical Papers, Springer, vol. 55(2), pages 349-374, May.
    5. Haeran Cho & Piotr Fryzlewicz, 2015. "Multiple-change-point detection for high dimensional time series via sparsified binary segmentation," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 77(2), pages 475-507, March.
    6. Bai, Jushan, 1997. "Estimating Multiple Breaks One at a Time," Econometric Theory, Cambridge University Press, vol. 13(3), pages 315-352, June.
    7. Davis, Richard A. & Lee, Thomas C.M. & Rodriguez-Yam, Gabriel A., 2006. "Structural Break Estimation for Nonstationary Time Series Models," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 223-239, March.
    8. Chao Du & Chu-Lan Michael Kao & S. C. Kou, 2016. "Stepwise Signal Extraction via Marginal Likelihood," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(513), pages 314-330, March.
    9. Pezzatti, Gianni B. & Zumbrunnen, Thomas & Bürgi, Matthias & Ambrosetti, Paolo & Conedera, Marco, 2013. "Fire regime shifts as a consequence of fire policy and socio-economic development: An analysis based on the change point approach," Forest Policy and Economics, Elsevier, vol. 29(C), pages 7-18.
    10. P. Fryzlewicz & S. Subba Rao, 2014. "Multiple-change-point detection for auto-regressive conditional heteroscedastic processes," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 76(5), pages 903-924, November.
    11. Robert Tibshirani & Michael Saunders & Saharon Rosset & Ji Zhu & Keith Knight, 2005. "Sparsity and smoothness via the fused lasso," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(1), pages 91-108, February.
    12. Bruce E. Hansen, 2001. "The New Econometrics of Structural Change: Dating Breaks in U.S. Labour Productivity," Journal of Economic Perspectives, American Economic Association, vol. 15(4), pages 117-128, Fall.
    13. Yao, Yi-Ching, 1988. "Estimating the number of change-points via Schwarz' criterion," Statistics & Probability Letters, Elsevier, vol. 6(3), pages 181-189, February.
    14. Lavielle, Marc, 1999. "Detection of multiple changes in a sequence of dependent variables," Stochastic Processes and their Applications, Elsevier, vol. 83(1), pages 79-102, September.
    15. Pan, Jianmin & Chen, Jiahua, 2006. "Application of modified information criterion to multiple change point problems," Journal of Multivariate Analysis, Elsevier, vol. 97(10), pages 2221-2241, November.
    16. Cho, Haeran & Fryzlewicz, Piotr, 2015. "Multiple-change-point detection for high dimensional time series via sparsified binary segmentation," LSE Research Online Documents on Economics 57147, London School of Economics and Political Science, LSE Library.
    17. Tengyao Wang & Richard J. Samworth, 2018. "High dimensional change point estimation via sparse projection," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 80(1), pages 57-83, January.
    18. Ciuperca, Gabriela, 2011. "A general criterion to determine the number of change-points," Statistics & Probability Letters, Elsevier, vol. 81(8), pages 1267-1275, August.
    19. Lee, Chung-Bow, 1995. "Estimating the number of change points in a sequence of independent normal random variables," Statistics & Probability Letters, Elsevier, vol. 25(3), pages 241-248, November.
    20. Nancy R. Zhang & David O. Siegmund, 2007. "A Modified Bayes Information Criterion with Applications to the Analysis of Comparative Genomic Hybridization Data," Biometrics, The International Biometric Society, vol. 63(1), pages 22-32, March.
    21. Fryzlewicz, Piotr, 2014. "Wild binary segmentation for multiple change-point detection," LSE Research Online Documents on Economics 57146, London School of Economics and Political Science, LSE Library.
    22. David S. Matteson & Nicholas A. James, 2014. "A Nonparametric Approach for Multiple Change Point Analysis of Multivariate Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(505), pages 334-345, March.
    23. Gabriela Ciuperca, 2014. "Erratum to: Model selection by LASSO methods in a change-point model," Statistical Papers, Springer, vol. 55(4), pages 1231-1232, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. S Kovács & P Bühlmann & H Li & A Munk, 2023. "Seeded binary segmentation: a general methodology for fast and optimal changepoint detection," Biometrika, Biometrika Trust, vol. 110(1), pages 249-256.
    2. Andreas Anastasiou & Piotr Fryzlewicz, 2022. "Detecting multiple generalized change-points by isolating single ones," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 85(2), pages 141-174, February.
    3. Cho, Haeran & Fryzlewicz, Piotr, 2023. "Multiple change point detection under serial dependence: wild contrast maximisation and gappy Schwarz algorithm," LSE Research Online Documents on Economics 120085, London School of Economics and Political Science, LSE Library.
    4. McGonigle, Euan T. & Cho, Haeran, 2023. "Robust multiscale estimation of time-average variance for time series segmentation," Computational Statistics & Data Analysis, Elsevier, vol. 179(C).
    5. Zifeng Zhao & Feiyu Jiang & Xiaofeng Shao, 2022. "Segmenting time series via self‐normalisation," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(5), pages 1699-1725, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Holger Dette & Theresa Eckle & Mathias Vetter, 2020. "Multiscale change point detection for dependent data," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 47(4), pages 1243-1274, December.
    2. Hajra Siddiqa & Sajid Ali & Ismail Shah, 2021. "Most recent changepoint detection in censored panel data," Computational Statistics, Springer, vol. 36(1), pages 515-540, March.
    3. Shi, Xuesheng & Gallagher, Colin & Lund, Robert & Killick, Rebecca, 2022. "A comparison of single and multiple changepoint techniques for time series data," Computational Statistics & Data Analysis, Elsevier, vol. 170(C).
    4. Ma, Chenchen & Tu, Yundong, 2023. "Group fused Lasso for large factor models with multiple structural breaks," Journal of Econometrics, Elsevier, vol. 233(1), pages 132-154.
    5. Pang, Tianxiao & Du, Lingjie & Chong, Terence Tai-Leung, 2021. "Estimating multiple breaks in nonstationary autoregressive models," Journal of Econometrics, Elsevier, vol. 221(1), pages 277-311.
    6. Marie Hušková & Zuzana Prášková, 2014. "Comments on: Extensions of some classical methods in change point analysis," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 23(2), pages 265-269, June.
    7. Wu Wang & Xuming He & Zhongyi Zhu, 2020. "Statistical inference for multiple change‐point models," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 47(4), pages 1149-1170, December.
    8. Alessandro Casini & Pierre Perron, 2018. "Structural Breaks in Time Series," Boston University - Department of Economics - Working Papers Series WP2019-02, Boston University - Department of Economics.
    9. Ardia, David & Dufays, Arnaud & Ordás Criado, Carlos, 2023. "Linking Frequentist and Bayesian Change-Point Methods," MPRA Paper 119486, University Library of Munich, Germany.
    10. Venkata Jandhyala & Stergios Fotopoulos & Ian MacNeill & Pengyu Liu, 2013. "Inference for single and multiple change-points in time series," Journal of Time Series Analysis, Wiley Blackwell, vol. 34(4), pages 423-446, July.
    11. Florian Pein & Hannes Sieling & Axel Munk, 2017. "Heterogeneous change point inference," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(4), pages 1207-1227, September.
    12. Liu, Bin & Zhang, Xinsheng & Liu, Yufeng, 2022. "High dimensional change point inference: Recent developments and extensions," Journal of Multivariate Analysis, Elsevier, vol. 188(C).
    13. Mengjia Yu & Xiaohui Chen, 2021. "Finite sample change point inference and identification for high‐dimensional mean vectors," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(2), pages 247-270, April.
    14. Barigozzi, Matteo & Cho, Haeran & Fryzlewicz, Piotr, 2018. "Simultaneous multiple change-point and factor analysis for high-dimensional time series," Journal of Econometrics, Elsevier, vol. 206(1), pages 187-225.
    15. Schroeder, Anna Louise & Fryzlewicz, Piotr, 2013. "Adaptive trend estimation in financial time series via multiscale change-point-induced basis recovery," LSE Research Online Documents on Economics 54934, London School of Economics and Political Science, LSE Library.
    16. Davis, Richard A. & Hancock, Stacey A. & Yao, Yi-Ching, 2016. "On consistency of minimum description length model selection for piecewise autoregressions," Journal of Econometrics, Elsevier, vol. 194(2), pages 360-368.
    17. Qing Yang & Yu-Ning Li & Yi Zhang, 2020. "Change point detection for nonparametric regression under strongly mixing process," Statistical Papers, Springer, vol. 61(4), pages 1465-1506, August.
    18. Kurozumi, Eiji & Tuvaandorj, Purevdorj, 2011. "Model selection criteria in multivariate models with multiple structural changes," Journal of Econometrics, Elsevier, vol. 164(2), pages 218-238, October.
    19. Chen, Likai & Wang, Weining & Wu, Wei Biao, 2019. "Inference of Break-Points in High-Dimensional Time Series," IRTG 1792 Discussion Papers 2019-013, Humboldt University of Berlin, International Research Training Group 1792 "High Dimensional Nonstationary Time Series".
    20. Lu Shaochuan, 2023. "Scalable Bayesian Multiple Changepoint Detection via Auxiliary Uniformisation," International Statistical Review, International Statistical Institute, vol. 91(1), pages 88-113, April.

    More about this item

    Keywords

    segmentation; break detection; jump detection; randomized algorithms; adaptive algorithms; multiscale methods; EP/ L014246/1;
    All these keywords.

    JEL classification:

    • C1 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ehl:lserod:103430. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: LSERO Manager (email available below). General contact details of provider: https://edirc.repec.org/data/lsepsuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.