IDEAS home Printed from https://ideas.repec.org/a/spt/stecon/v9y2020i1f9_1_3.html
   My bibliography  Save this article

An L1 smoother for outlier cleaning of time series

Author

Listed:
  • Ilaria Lucrezia Amerise
  • Agostino Tarsitano

Abstract

This paper introduces a new robust outlier cleaner speciï¬ c for high-frequency time series data and provides guidelines for researchers who wish to use this procedure before the analysis process starts. The essence of the method is a fully automatic, data-driven procedure based on ï¬ tting, by least absolute deviations, a reference function to the actual time series. Once the reference curve has been deï¬ ned, it can be used to establish bands such that all observations that deviate from the reference curve by more than a preï¬ xed amount will be replaced. Properties of the new screening tool are investigated through the accuracy of simultaneous prediction intervals produced by Box-Jenkins models applied to real data, before and after the outlier cleaner usage. It is shown that the new method can be validly used as a data preparation technique to ensure that statistical analysis is supported by clear-cut data.Mathematics Subject Classiï¬ cation: 90C05, 62M20, 37M10 Keywords: Linear programming, simultaneous prediction intervals, electricity prices, pre-processing time series.

Suggested Citation

  • Ilaria Lucrezia Amerise & Agostino Tarsitano, 2020. "An L1 smoother for outlier cleaning of time series," Journal of Statistical and Econometric Methods, SCIENPRESS Ltd, vol. 9(1), pages 1-3.
    • Handle: RePEc:spt:stecon:v:9:y:2020:i:1:f:9_1_3
    as

    Download full text from publisher

    File URL: http://www.scienpress.com/Upload/JSEM%2fVol%209_1_3.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Siu Hung Cheung & Ka Ho Wu & Wai Sum Chan, 1998. "Simultaneous prediction intervals for autoregressive-integrated moving-average models: A comparative study," Computational Statistics & Data Analysis, Elsevier, vol. 28(3), pages 297-306, September.
    2. Ledolter, Johannes, 1989. "The effect of additive outliers on the forecasts from ARIMA models," International Journal of Forecasting, Elsevier, vol. 5(2), pages 231-240.
    3. Weron, Rafał & Zator, Michał, 2015. "A note on using the Hodrick–Prescott filter in electricity markets," Energy Economics, Elsevier, vol. 48(C), pages 1-6.
    4. Bigerna, Simona & Bollino, Carlo Andrea, 2016. "Ramsey prices in the Italian electricity market," Energy Policy, Elsevier, vol. 88(C), pages 603-612.
    5. Gianfreda, Angelica & Grossi, Luigi, 2012. "Forecasting Italian electricity zonal prices with exogenous variables," Energy Economics, Elsevier, vol. 34(6), pages 2228-2239.
    6. Li, Johnny Siu-Hang & Chan, Wai-Sum, 2011. "Time-simultaneous prediction bands: A new look at the uncertainty involved in forecasting mortality," Insurance: Mathematics and Economics, Elsevier, vol. 49(1), pages 81-88, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fianu, Emmanuel Senyo & Ahelegbey, Daniel Felix & Grossi, Luigi, 2022. "Modeling risk contagion in the Italian zonal electricity market," European Journal of Operational Research, Elsevier, vol. 298(2), pages 656-679.
    2. Weron, Rafał, 2014. "Electricity price forecasting: A review of the state-of-the-art with a look into the future," International Journal of Forecasting, Elsevier, vol. 30(4), pages 1030-1081.
    3. Sapio, Alessandro & Spagnolo, Nicola, 2020. "The effect of a new power cable on energy prices volatility spillovers," Energy Policy, Elsevier, vol. 144(C).
    4. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    5. Sapio, Alessandro, 2019. "Greener, more integrated, and less volatile? A quantile regression analysis of Italian wholesale electricity prices," Energy Policy, Elsevier, vol. 126(C), pages 452-469.
    6. Angelica Gianfreda & Francesco Ravazzolo & Luca Rossini, 2023. "Large Time‐Varying Volatility Models for Hourly Electricity Prices," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 85(3), pages 545-573, June.
    7. Doornik, Jurgen A. & Ooms, Marius, 2008. "Multimodality in GARCH regression models," International Journal of Forecasting, Elsevier, vol. 24(3), pages 432-448.
    8. Nowotarski, Jakub & Tomczyk, Jakub & Weron, Rafał, 2013. "Robust estimation and forecasting of the long-term seasonal component of electricity spot prices," Energy Economics, Elsevier, vol. 39(C), pages 13-27.
    9. Billé, Anna Gloria & Gianfreda, Angelica & Del Grosso, Filippo & Ravazzolo, Francesco, 2023. "Forecasting electricity prices with expert, linear, and nonlinear models," International Journal of Forecasting, Elsevier, vol. 39(2), pages 570-586.
    10. Marcjasz, Grzegorz & Uniejewski, Bartosz & Weron, Rafał, 2019. "On the importance of the long-term seasonal component in day-ahead electricity price forecasting with NARX neural networks," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1520-1532.
    11. Grané, Aurea & Veiga, Helena, 2010. "Outliers in Garch models and the estimation of risk measures," DES - Working Papers. Statistics and Econometrics. WS ws100502, Universidad Carlos III de Madrid. Departamento de Estadística.
    12. Zachmann, Georg, 2013. "A stochastic fuel switching model for electricity prices," Energy Economics, Elsevier, vol. 35(C), pages 5-13.
    13. Jussi Tolvi, 2001. "Outliers in eleven Finnish macroeconomic time series," Finnish Economic Papers, Finnish Economic Association, vol. 14(1), pages 14-32, Spring.
    14. Weron, Rafał & Zator, Michał, 2015. "A note on using the Hodrick–Prescott filter in electricity markets," Energy Economics, Elsevier, vol. 48(C), pages 1-6.
    15. Bianco, Vincenzo & Scarpa, Federico & Tagliafico, Luca A., 2015. "Long term outlook of primary energy consumption of the Italian thermoelectric sector: Impact of fuel and carbon prices," Energy, Elsevier, vol. 87(C), pages 153-164.
    16. Francesco Battaglia & Lia Orfei, 2005. "Outlier Detection And Estimation In NonLinear Time Series," Journal of Time Series Analysis, Wiley Blackwell, vol. 26(1), pages 107-121, January.
    17. Blake, David & El Karoui, Nicole & Loisel, Stéphane & MacMinn, Richard, 2018. "Longevity risk and capital markets: The 2015–16 update," Insurance: Mathematics and Economics, Elsevier, vol. 78(C), pages 157-173.
    18. Nowotarski, Jakub & Weron, Rafał, 2018. "Recent advances in electricity price forecasting: A review of probabilistic forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 81(P1), pages 1548-1568.
    19. H. W. Wayne Yang & Po-Wei Shen & An-Sing Chen, 2020. "Trimming Effects And Momentum Investing," The International Journal of Business and Finance Research, The Institute for Business and Finance Research, vol. 14(2), pages 73-87.
    20. Auer, Benjamin R., 2016. "How does Germany's green energy policy affect electricity market volatility? An application of conditional autoregressive range models," Energy Policy, Elsevier, vol. 98(C), pages 621-628.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spt:stecon:v:9:y:2020:i:1:f:9_1_3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Eleftherios Spyromitros-Xioufis (email available below). General contact details of provider: http://www.scienpress.com/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.