IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v236y2019icp196-210.html
   My bibliography  Save this article

On the impact of outlier filtering on the electricity price forecasting accuracy

Author

Listed:
  • Afanasyev, Dmitriy O.
  • Fedorova, Elena A.

Abstract

Increasing the accuracy of short-term electricity price forecasting allows day-ahead power market participants to obtain a positive economic effect by bidding close to the equilibrium price. However the electricity price time-series is generally infested with extreme values due to high price volatility. This paper discusses the impact of outlier filtering on forecasting accuracy based on a recently introduced seasonal component autoregressive model. We consider such methods of outlier detection (with a priori defined cut-off parameter) as threshold, standard deviation, percentage, recursive, and moving filter on prices. It is shown that such data pre-processing often leads to the forecasting accuracy gain while the error decrease (relative to the approach without filtering) in a number of cases may reach 1.8–1.9% of the average weekly price (in absolute values). For an a priori defined cut-off parameter, the simple threshold and standard deviation filter on prices outperform other considered methods, and yield to the accuracy gain in 63% and 67% of cases, correspondingly. At the same time, in case of the out-of-sample filter parameter grid-optimization all of the methods demonstrate comparable prediction power (equal to the marginal performance). But, practically speaking, such optimization is time-consuming and cannot be carried out on unavailable future data. As an competitive alternative, we propose a combined filter on prices based on a committee machine which uses the results of individual non-optimized algorithms and is not time-consuming, but gives accuracy comparable to the best one obtained for each of the studied electricity markets and leads to forecast gain in 63% of the considered cases.

Suggested Citation

  • Afanasyev, Dmitriy O. & Fedorova, Elena A., 2019. "On the impact of outlier filtering on the electricity price forecasting accuracy," Applied Energy, Elsevier, vol. 236(C), pages 196-210.
  • Handle: RePEc:eee:appene:v:236:y:2019:i:c:p:196-210
    DOI: 10.1016/j.apenergy.2018.11.076
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261918317926
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.apenergy.2018.11.076?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Karakatsani, Nektaria V. & Bunn, Derek W., 2008. "Forecasting electricity prices: The impact of fundamentals and time-varying coefficients," International Journal of Forecasting, Elsevier, vol. 24(4), pages 764-785.
    2. Crespo Cuaresma, Jesús & Hlouskova, Jaroslava & Kossmeier, Stephan & Obersteiner, Michael, 2004. "Forecasting electricity spot-prices using linear univariate time-series models," Applied Energy, Elsevier, vol. 77(1), pages 87-106, January.
    3. Alvaro Cartea & Marcelo Figueroa, 2005. "Pricing in Electricity Markets: A Mean Reverting Jump Diffusion Model with Seasonality," Applied Mathematical Finance, Taylor & Francis Journals, vol. 12(4), pages 313-335.
    4. Misiorek Adam & Trueck Stefan & Weron Rafal, 2006. "Point and Interval Forecasting of Spot Electricity Prices: Linear vs. Non-Linear Time Series Models," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 10(3), pages 1-36, September.
    5. Bordignon, Silvano & Bunn, Derek W. & Lisi, Francesco & Nan, Fany, 2013. "Combining day-ahead forecasts for British electricity prices," Energy Economics, Elsevier, vol. 35(C), pages 88-103.
    6. Fanone, Enzo & Gamba, Andrea & Prokopczuk, Marcel, 2013. "The case of negative day-ahead electricity prices," Energy Economics, Elsevier, vol. 35(C), pages 22-34.
    7. Lisi, Francesco & Nan, Fany, 2014. "Component estimation for electricity prices: Procedures and comparisons," Energy Economics, Elsevier, vol. 44(C), pages 143-159.
    8. Janczura, Joanna & Trück, Stefan & Weron, Rafał & Wolff, Rodney C., 2013. "Identifying spikes and seasonal components in electricity spot price data: A guide to robust modeling," Energy Economics, Elsevier, vol. 38(C), pages 96-110.
    9. Bierbrauer, Michael & Menn, Christian & Rachev, Svetlozar T. & Truck, Stefan, 2007. "Spot and derivative pricing in the EEX power market," Journal of Banking & Finance, Elsevier, vol. 31(11), pages 3462-3485, November.
    10. Weron, Rafal & Misiorek, Adam, 2008. "Forecasting spot electricity prices: A comparison of parametric and semiparametric time series models," International Journal of Forecasting, Elsevier, vol. 24(4), pages 744-763.
    11. De Jong Cyriel, 2006. "The Nature of Power Spikes: A Regime-Switch Approach," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 10(3), pages 1-28, September.
    12. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    13. Weron, Rafal, 2008. "Market price of risk implied by Asian-style electricity options and futures," Energy Economics, Elsevier, vol. 30(3), pages 1098-1115, May.
    14. Xiao, Liye & Shao, Wei & Yu, Mengxia & Ma, Jing & Jin, Congjun, 2017. "Research and application of a hybrid wavelet neural network model with the improved cuckoo search algorithm for electrical power system forecasting," Applied Energy, Elsevier, vol. 198(C), pages 203-222.
    15. Peter R. Hansen & Asger Lunde & James M. Nason, 2011. "The Model Confidence Set," Econometrica, Econometric Society, vol. 79(2), pages 453-497, March.
    16. Keles, Dogan & Scelle, Jonathan & Paraschiv, Florentina & Fichtner, Wolf, 2016. "Extended forecast methods for day-ahead electricity spot prices applying artificial neural networks," Applied Energy, Elsevier, vol. 162(C), pages 218-230.
    17. Afanasyev, Dmitriy O. & Fedorova, Elena A., 2016. "The long-term trends on the electricity markets: Comparison of empirical mode and wavelet decompositions," Energy Economics, Elsevier, vol. 56(C), pages 432-442.
    18. Nowotarski, Jakub & Tomczyk, Jakub & Weron, Rafał, 2013. "Robust estimation and forecasting of the long-term seasonal component of electricity spot prices," Energy Economics, Elsevier, vol. 39(C), pages 13-27.
    19. Janczura, Joanna & Weron, Rafal, 2010. "An empirical comparison of alternate regime-switching models for electricity spot prices," Energy Economics, Elsevier, vol. 32(5), pages 1059-1073, September.
    20. Garcia, Márcio G.P. & Medeiros, Marcelo C. & Vasconcelos, Gabriel F.R., 2017. "Real-time inflation forecasting with high-dimensional models: The case of Brazil," International Journal of Forecasting, Elsevier, vol. 33(3), pages 679-693.
    21. Rafal Weron, 2006. "Modeling and Forecasting Electricity Loads and Prices: A Statistical Approach," HSC Books, Hugo Steinhaus Center, Wroclaw University of Technology, number hsbook0601, December.
    22. Trueck, Stefan & Weron, Rafal & Wolff, Rodney, 2007. "Outlier Treatment and Robust Approaches for Modeling Electricity Spot Prices," MPRA Paper 4711, University Library of Munich, Germany.
    23. Nowotarski, Jakub & Weron, Rafał, 2016. "On the importance of the long-term seasonal component in day-ahead electricity price forecasting," Energy Economics, Elsevier, vol. 57(C), pages 228-235.
    24. Peter Reinhard Hansen & Asger Lunde & James M. Nason, 2005. "Model confidence sets for forecasting models," FRB Atlanta Working Paper 2005-07, Federal Reserve Bank of Atlanta.
    25. Samuels, Jon D. & Sekkel, Rodrigo M., 2017. "Model Confidence Sets and forecast combination," International Journal of Forecasting, Elsevier, vol. 33(1), pages 48-60.
    26. Nowotarski, Jakub & Raviv, Eran & Trück, Stefan & Weron, Rafał, 2014. "An empirical comparison of alternative schemes for combining electricity spot price forecasts," Energy Economics, Elsevier, vol. 46(C), pages 395-412.
    27. Sergey Voronin & Jarmo Partanen, 2013. "Price Forecasting in the Day-Ahead Energy Market by an Iterative Method with Separate Normal Price and Price Spike Frameworks," Energies, MDPI, Open Access Journal, vol. 6(11), pages 1-24, November.
    28. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
    29. Weron, Rafał, 2014. "Electricity price forecasting: A review of the state-of-the-art with a look into the future," International Journal of Forecasting, Elsevier, vol. 30(4), pages 1030-1081.
    30. Yang, Zhang & Ce, Li & Lian, Li, 2017. "Electricity price forecasting by a hybrid model, combining wavelet transform, ARMA and kernel-based extreme learning machine methods," Applied Energy, Elsevier, vol. 190(C), pages 291-305.
    31. Tan, Zhongfu & Zhang, Jinliang & Wang, Jianhui & Xu, Jun, 2010. "Day-ahead electricity price forecasting using wavelet transform combined with ARIMA and GARCH models," Applied Energy, Elsevier, vol. 87(11), pages 3606-3610, November.
    32. He, Kaijian & Yu, Lean & Tang, Ling, 2015. "Electricity price forecasting with a BED (Bivariate EMD Denoising) methodology," Energy, Elsevier, vol. 91(C), pages 601-609.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gang Zhou & Zhongjie Han & Jin Fu & Guan Hua Xu & Chengjin Ye, 2020. "Iterative Online Fault Identification Scheme for High-Voltage Circuit Breaker Utilizing a Lost Data Repair Technique," Energies, MDPI, Open Access Journal, vol. 13(13), pages 1-15, June.
    2. Uniejewski, Bartosz & Weron, Rafał, 2021. "Regularized quantile regression averaging for probabilistic electricity price forecasting," Energy Economics, Elsevier, vol. 95(C).
    3. Alex Borodin & Irina Mityushina & Elena Streltsova & Andrey Kulikov & Irina Yakovenko & Anzhela Namitulina, 2021. "Mathematical Modeling for Financial Analysis of an Enterprise: Motivating of Not Open Innovation," Journal of Open Innovation: Technology, Market, and Complexity, MDPI, Open Access Journal, vol. 7(1), pages 1-15, March.
    4. Qiao, Weibiao & Yang, Zhe, 2020. "Forecast the electricity price of U.S. using a wavelet transform-based hybrid model," Energy, Elsevier, vol. 193(C).
    5. Arkadiusz Jędrzejewski & Grzegorz Marcjasz & Rafał Weron, 2021. "Importance of the Long-Term Seasonal Component in Day-Ahead Electricity Price Forecasting Revisited: Parameter-Rich Models Estimated via the LASSO," Energies, MDPI, Open Access Journal, vol. 14(11), pages 1-17, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Weron, Rafał, 2014. "Electricity price forecasting: A review of the state-of-the-art with a look into the future," International Journal of Forecasting, Elsevier, vol. 30(4), pages 1030-1081.
    2. Nowotarski, Jakub & Weron, Rafał, 2018. "Recent advances in electricity price forecasting: A review of probabilistic forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 81(P1), pages 1548-1568.
    3. Ziel, Florian & Weron, Rafał, 2018. "Day-ahead electricity price forecasting with high-dimensional structures: Univariate vs. multivariate modeling frameworks," Energy Economics, Elsevier, vol. 70(C), pages 396-420.
    4. Florian Ziel & Rafal Weron, 2016. "Day-ahead electricity price forecasting with high-dimensional structures: Univariate vs. multivariate models," HSC Research Reports HSC/16/08, Hugo Steinhaus Center, Wroclaw University of Technology.
    5. Rafal Weron & Florian Ziel, 2018. "Electricity price forecasting," HSC Research Reports HSC/18/08, Hugo Steinhaus Center, Wroclaw University of Technology.
    6. Grzegorz Marcjasz & Bartosz Uniejewski & Rafal Weron, 2017. "Importance of the long-term seasonal component in day-ahead electricity price forecasting revisited: Neural network models," HSC Research Reports HSC/17/03, Hugo Steinhaus Center, Wroclaw University of Technology.
    7. Marcjasz, Grzegorz & Uniejewski, Bartosz & Weron, Rafał, 2019. "On the importance of the long-term seasonal component in day-ahead electricity price forecasting with NARX neural networks," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1520-1532.
    8. Weron, Rafał & Zator, Michał, 2015. "A note on using the Hodrick–Prescott filter in electricity markets," Energy Economics, Elsevier, vol. 48(C), pages 1-6.
    9. Avci, Ezgi & Ketter, Wolfgang & van Heck, Eric, 2018. "Managing electricity price modeling risk via ensemble forecasting: The case of Turkey," Energy Policy, Elsevier, vol. 123(C), pages 390-403.
    10. Marcjasz, Grzegorz & Uniejewski, Bartosz & Weron, Rafał, 2020. "Probabilistic electricity price forecasting with NARX networks: Combine point or probabilistic forecasts?," International Journal of Forecasting, Elsevier, vol. 36(2), pages 466-479.
    11. Grossi, Luigi & Nan, Fany, 2019. "Robust forecasting of electricity prices: Simulations, models and the impact of renewable sources," Technological Forecasting and Social Change, Elsevier, vol. 141(C), pages 305-318.
    12. Bartosz Uniejewski & Jakub Nowotarski & Rafał Weron, 2016. "Automated Variable Selection and Shrinkage for Day-Ahead Electricity Price Forecasting," Energies, MDPI, Open Access Journal, vol. 9(8), pages 1-22, August.
    13. Grzegorz Marcjasz & Tomasz Serafin & Rafał Weron, 2018. "Selection of Calibration Windows for Day-Ahead Electricity Price Forecasting," Energies, MDPI, Open Access Journal, vol. 11(9), pages 1-20, September.
    14. Nowotarski, Jakub & Weron, Rafał, 2016. "On the importance of the long-term seasonal component in day-ahead electricity price forecasting," Energy Economics, Elsevier, vol. 57(C), pages 228-235.
    15. Uniejewski, Bartosz & Marcjasz, Grzegorz & Weron, Rafał, 2019. "On the importance of the long-term seasonal component in day-ahead electricity price forecasting: Part II — Probabilistic forecasting," Energy Economics, Elsevier, vol. 79(C), pages 171-182.
    16. Luigi Grossi & Fany Nan, 2018. "The influence of renewables on electricity price forecasting: a robust approach," Working Papers 2018/10, Institut d'Economia de Barcelona (IEB).
    17. Luigi Grossi & Fany Nan, 2017. "Forecasting electricity prices through robust nonlinear models," Working Papers 06/2017, University of Verona, Department of Economics.
    18. Nowotarski, Jakub & Tomczyk, Jakub & Weron, Rafał, 2013. "Robust estimation and forecasting of the long-term seasonal component of electricity spot prices," Energy Economics, Elsevier, vol. 39(C), pages 13-27.
    19. Bartosz Uniejewski & Rafał Weron, 2018. "Efficient Forecasting of Electricity Spot Prices with Expert and LASSO Models," Energies, MDPI, Open Access Journal, vol. 11(8), pages 1-26, August.
    20. Alexandre Lucas & Konstantinos Pegios & Evangelos Kotsakis & Dan Clarke, 2020. "Price Forecasting for the Balancing Energy Market Using Machine-Learning Regression," Energies, MDPI, Open Access Journal, vol. 13(20), pages 1-16, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:236:y:2019:i:c:p:196-210. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.