IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v13y2020i20p5420-d430252.html
   My bibliography  Save this article

Price Forecasting for the Balancing Energy Market Using Machine-Learning Regression

Author

Listed:
  • Alexandre Lucas

    (Directorate C—Energy, Transport and Climate, Joint Research Centre (JRC), European Commission, Via E. Fermi 2749, 21027 Ispra (V.A.), Italy)

  • Konstantinos Pegios

    (Kiwi Power, Innovation and Grants Programme, 3 London Wall Buildings, London Wall, London EC2M 5PD, UK)

  • Evangelos Kotsakis

    (Directorate C—Energy, Transport and Climate, Joint Research Centre (JRC), European Commission, Via E. Fermi 2749, 21027 Ispra (V.A.), Italy)

  • Dan Clarke

    (Kiwi Power, Innovation and Grants Programme, 3 London Wall Buildings, London Wall, London EC2M 5PD, UK)

Abstract

The importance of price forecasting has gained attention over the last few years, with the growth of aggregators and the general opening of the European electricity markets. Market participants manage a tradeoff between, bidding in a lower price market (day-ahead), but with typically higher volume, or aiming for a lower volume market but with potentially higher returns (balance energy market). Companies try to forecast the extremes of revenues or prices, in order to manage risk and opportunity, assigning their assets in an optimal way. It is thought that in general, electricity markets have quasi-deterministic principles, rather than being based on speculation, hence the desire to forecast the price based on variables that can describe the outcome of the market. Many studies address this problem from a statistical approach or by performing multiple-variable regressions, but they very often focus only on the time series analysis. In 2019, the Loss of Load Probability (LOLP) was made available in the UK for the first time. Taking this opportunity, this study focusses on five LOLP variables (with different time-ahead estimations) and other quasi-deterministic variables, to explain the price behavior of a multi-variable regression model. These include base production, system load, solar and wind generation, seasonality, day-ahead price and imbalance volume contributions. Three machine-learning algorithms were applied to test for performance, Gradient Boosting (GB), Random Forest (RF) and XGBoost. XGBoost presented higher performance and so it was chosen for the implementation of the real time forecast step. The model returns a Mean Absolute Error (MAE) of 7.89 £/MWh, a coefficient of determination (R2 score) of 76.8% and a Mean Squared Error (MSE) of 124.74. The variables that contribute the most to the model are the Net Imbalance Volume, the LOLP (aggregated), the month and the De-rated margins (aggregated) with 28.6%, 27.5%, 14.0%, and 8.9% of weight on feature importance respectively.

Suggested Citation

  • Alexandre Lucas & Konstantinos Pegios & Evangelos Kotsakis & Dan Clarke, 2020. "Price Forecasting for the Balancing Energy Market Using Machine-Learning Regression," Energies, MDPI, vol. 13(20), pages 1-16, October.
    • Handle: RePEc:gam:jeners:v:13:y:2020:i:20:p:5420-:d:430252
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/13/20/5420/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/13/20/5420/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ziel, Florian & Steinert, Rick & Husmann, Sven, 2015. "Efficient modeling and forecasting of electricity spot prices," Energy Economics, Elsevier, vol. 47(C), pages 98-111.
    2. Misiorek Adam & Trueck Stefan & Weron Rafal, 2006. "Point and Interval Forecasting of Spot Electricity Prices: Linear vs. Non-Linear Time Series Models," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 10(3), pages 1-36, September.
    3. Helyette Geman & A. Roncoroni, 2006. "Understanding the Fine Structure of Electricity Prices," Post-Print halshs-00144198, HAL.
    4. Panapakidis, Ioannis P. & Dagoumas, Athanasios S., 2016. "Day-ahead electricity price forecasting via the application of artificial neural network based models," Applied Energy, Elsevier, vol. 172(C), pages 132-151.
    5. Lanne, Markku & Lütkepohl, Helmut, 2010. "Structural Vector Autoregressions With Nonnormal Residuals," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(1), pages 159-168.
    6. Alvaro Cartea & Marcelo Figueroa, 2005. "Pricing in Electricity Markets: A Mean Reverting Jump Diffusion Model with Seasonality," Applied Mathematical Finance, Taylor & Francis Journals, vol. 12(4), pages 313-335.
    7. Weron, Rafał, 2014. "Electricity price forecasting: A review of the state-of-the-art with a look into the future," International Journal of Forecasting, Elsevier, vol. 30(4), pages 1030-1081.
    8. Magdalena Borgosz-Koczwara & Aleksander Weron & Agnieszka Wyłomańska, 2009. "Stochastic models for bidding strategies on oligopoly electricity market," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 69(3), pages 579-592, July.
    9. Karakatsani, Nektaria V. & Bunn, Derek W., 2008. "Forecasting electricity prices: The impact of fundamentals and time-varying coefficients," International Journal of Forecasting, Elsevier, vol. 24(4), pages 764-785.
    10. Keles, Dogan & Scelle, Jonathan & Paraschiv, Florentina & Fichtner, Wolf, 2016. "Extended forecast methods for day-ahead electricity spot prices applying artificial neural networks," Applied Energy, Elsevier, vol. 162(C), pages 218-230.
    11. Holmberg, Pär & Newbery, David & Ralph, Daniel, 2013. "Supply function equilibria: Step functions and continuous representations," Journal of Economic Theory, Elsevier, vol. 148(4), pages 1509-1551.
    12. Trueck, Stefan & Weron, Rafal & Wolff, Rodney, 2007. "Outlier Treatment and Robust Approaches for Modeling Electricity Spot Prices," MPRA Paper 4711, University Library of Munich, Germany.
    13. Weron, Rafal & Misiorek, Adam, 2008. "Forecasting spot electricity prices: A comparison of parametric and semiparametric time series models," International Journal of Forecasting, Elsevier, vol. 24(4), pages 744-763.
    14. Brijs, Tom & De Vos, Kristof & De Jonghe, Cedric & Belmans, Ronnie, 2015. "Statistical analysis of negative prices in European balancing markets," Renewable Energy, Elsevier, vol. 80(C), pages 53-60.
    15. Kowalska-Pyzalska, Anna & Maciejowska, Katarzyna & Suszczyński, Karol & Sznajd-Weron, Katarzyna & Weron, Rafał, 2014. "Turning green: Agent-based modeling of the adoption of dynamic electricity tariffs," Energy Policy, Elsevier, vol. 72(C), pages 164-174.
    16. Lanne, Markku & Lütkepohl, Helmut & Maciejowska, Katarzyna, 2010. "Structural vector autoregressions with Markov switching," Journal of Economic Dynamics and Control, Elsevier, vol. 34(2), pages 121-131, February.
    17. Janczura, Joanna & Trück, Stefan & Weron, Rafał & Wolff, Rodney C., 2013. "Identifying spikes and seasonal components in electricity spot price data: A guide to robust modeling," Energy Economics, Elsevier, vol. 38(C), pages 96-110.
    18. Janczura, Joanna & Weron, Rafal, 2010. "An empirical comparison of alternate regime-switching models for electricity spot prices," Energy Economics, Elsevier, vol. 32(5), pages 1059-1073, September.
    19. Bierbrauer, Michael & Menn, Christian & Rachev, Svetlozar T. & Truck, Stefan, 2007. "Spot and derivative pricing in the EEX power market," Journal of Banking & Finance, Elsevier, vol. 31(11), pages 3462-3485, November.
    20. Albanese, Claudio & Lo, Harry & Tompaidis, Stathis, 2012. "A numerical algorithm for pricing electricity derivatives for jump-diffusion processes based on continuous time lattices," European Journal of Operational Research, Elsevier, vol. 222(2), pages 361-368.
    21. Adam Misiorek & Rafal Weron, 2006. "Interval forecasting of spot electricity prices," HSC Research Reports HSC/06/05, Hugo Steinhaus Center, Wroclaw University of Technology.
    22. Weron, Rafał & Zator, Michał, 2014. "Revisiting the relationship between spot and futures prices in the Nord Pool electricity market," Energy Economics, Elsevier, vol. 44(C), pages 178-190.
    23. Unknown, 2005. "Forward," 2005 Conference: Slovenia in the EU - Challenges for Agriculture, Food Science and Rural Affairs, November 10-11, 2005, Moravske Toplice, Slovenia 183804, Slovenian Association of Agricultural Economists (DAES).
    24. Fanone, Enzo & Gamba, Andrea & Prokopczuk, Marcel, 2013. "The case of negative day-ahead electricity prices," Energy Economics, Elsevier, vol. 35(C), pages 22-34.
    25. Tryggvi Jónsson & Pierre Pinson & Henrik Aa. Nielsen & Henrik Madsen, 2014. "Exponential Smoothing Approaches for Prediction in Real-Time Electricity Markets," Energies, MDPI, vol. 7(6), pages 1-23, June.
    26. Orhan Altuğ Karabiber & George Xydis, 2019. "Electricity Price Forecasting in the Danish Day-Ahead Market Using the TBATS, ANN and ARIMA Methods," Energies, MDPI, vol. 12(5), pages 1-29, March.
    27. repec:dau:papers:123456789/1433 is not listed on IDEAS
    28. Hélyette Geman & Andrea Roncoroni, 2006. "Understanding the Fine Structure of Electricity Prices," The Journal of Business, University of Chicago Press, vol. 79(3), pages 1225-1262, May.
    29. Katarzyna Maciejowska, 2014. "Fundamental and speculative shocks, what drives electricity prices?," HSC Research Reports HSC/14/05, Hugo Steinhaus Center, Wroclaw University of Technology.
    30. Rafal Weron, 2006. "Modeling and Forecasting Electricity Loads and Prices: A Statistical Approach," HSC Books, Hugo Steinhaus Center, Wroclaw University of Technology, number hsbook0601.
    31. Ping-Huan Kuo & Chiou-Jye Huang, 2018. "An Electricity Price Forecasting Model by Hybrid Structured Deep Neural Networks," Sustainability, MDPI, vol. 10(4), pages 1-17, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Florian Marcel Nuţă & Alina Cristina Nuţă & Cristina Gabriela Zamfir & Stefan-Mihai Petrea & Dan Munteanu & Dragos Sebastian Cristea, 2021. "National Carbon Accounting—Analyzing the Impact of Urbanization and Energy-Related Factors upon CO 2 Emissions in Central–Eastern European Countries by Using Machine Learning Algorithms and Panel Data," Energies, MDPI, vol. 14(10), pages 1-23, May.
    2. Michał Narajewski, 2022. "Probabilistic Forecasting of German Electricity Imbalance Prices," Energies, MDPI, vol. 15(14), pages 1-17, July.
    3. Sheybanivaziri, Samaneh & Le Dréau, Jérôme & Kazmi, Hussain, 2024. "Forecasting price spikes in day-ahead electricity markets: techniques, challenges, and the road ahead," Discussion Papers 2024/1, Norwegian School of Economics, Department of Business and Management Science.
    4. Sinan Deng & John Inekwe & Vladimir Smirnov & Andrew Wait & Chao Wang, 2023. "Machine Learning and Deep Learning Forecasts of Electricity Imbalance Prices," Working Papers 2023-03, University of Sydney, School of Economics.
    5. Micha{l} Narajewski, 2022. "Probabilistic forecasting of German electricity imbalance prices," Papers 2205.11439, arXiv.org.
    6. Stefano Frizzo Stefenon & Laio Oriel Seman & Viviana Cocco Mariani & Leandro dos Santos Coelho, 2023. "Aggregating Prophet and Seasonal Trend Decomposition for Time Series Forecasting of Italian Electricity Spot Prices," Energies, MDPI, vol. 16(3), pages 1-18, January.
    7. Miguel A. Jaramillo-Morán & Daniel Fernández-Martínez & Agustín García-García & Diego Carmona-Fernández, 2021. "Improving Artificial Intelligence Forecasting Models Performance with Data Preprocessing: European Union Allowance Prices Case Study," Energies, MDPI, vol. 14(23), pages 1-23, November.
    8. Conor Lynch & Christian O’Leary & Preetham Govind Kolar Sundareshan & Yavuz Akin, 2021. "Experimental Analysis of GBM to Expand the Time Horizon of Irish Electricity Price Forecasts," Energies, MDPI, vol. 14(22), pages 1-11, November.
    9. Silvia Golia & Luigi Grossi & Matteo Pelagatti, 2022. "Machine Learning Models and Intra-Daily Market Information for the Prediction of Italian Electricity Prices," Forecasting, MDPI, vol. 5(1), pages 1-21, December.
    10. Yuriy Bilan & Serhiy Kozmenko & Alex Plastun, 2022. "Price Forecasting in Energy Market," Energies, MDPI, vol. 15(24), pages 1-6, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Weron, Rafał, 2014. "Electricity price forecasting: A review of the state-of-the-art with a look into the future," International Journal of Forecasting, Elsevier, vol. 30(4), pages 1030-1081.
    2. Rafal Weron & Florian Ziel, 2018. "Electricity price forecasting," HSC Research Reports HSC/18/08, Hugo Steinhaus Center, Wroclaw University of Technology.
    3. Afanasyev, Dmitriy O. & Fedorova, Elena A., 2019. "On the impact of outlier filtering on the electricity price forecasting accuracy," Applied Energy, Elsevier, vol. 236(C), pages 196-210.
    4. Avci, Ezgi & Ketter, Wolfgang & van Heck, Eric, 2018. "Managing electricity price modeling risk via ensemble forecasting: The case of Turkey," Energy Policy, Elsevier, vol. 123(C), pages 390-403.
    5. Janczura, Joanna & Trück, Stefan & Weron, Rafał & Wolff, Rodney C., 2013. "Identifying spikes and seasonal components in electricity spot price data: A guide to robust modeling," Energy Economics, Elsevier, vol. 38(C), pages 96-110.
    6. Mayer, Klaus & Trück, Stefan, 2018. "Electricity markets around the world," Journal of Commodity Markets, Elsevier, vol. 9(C), pages 77-100.
    7. Ziel, Florian & Weron, Rafał, 2018. "Day-ahead electricity price forecasting with high-dimensional structures: Univariate vs. multivariate modeling frameworks," Energy Economics, Elsevier, vol. 70(C), pages 396-420.
    8. Florian Ziel & Rafal Weron, 2016. "Day-ahead electricity price forecasting with high-dimensional structures: Univariate vs. multivariate models," HSC Research Reports HSC/16/08, Hugo Steinhaus Center, Wroclaw University of Technology.
    9. Nowotarski, Jakub & Tomczyk, Jakub & Weron, Rafał, 2013. "Robust estimation and forecasting of the long-term seasonal component of electricity spot prices," Energy Economics, Elsevier, vol. 39(C), pages 13-27.
    10. Umut Ugurlu & Ilkay Oksuz & Oktay Tas, 2018. "Electricity Price Forecasting Using Recurrent Neural Networks," Energies, MDPI, vol. 11(5), pages 1-23, May.
    11. Nowotarski, Jakub & Weron, Rafał, 2018. "Recent advances in electricity price forecasting: A review of probabilistic forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 81(P1), pages 1548-1568.
    12. Yan, Guan & Trück, Stefan, 2020. "A dynamic network analysis of spot electricity prices in the Australian national electricity market," Energy Economics, Elsevier, vol. 92(C).
    13. Bartosz Uniejewski & Jakub Nowotarski & Rafał Weron, 2016. "Automated Variable Selection and Shrinkage for Day-Ahead Electricity Price Forecasting," Energies, MDPI, vol. 9(8), pages 1-22, August.
    14. Nowotarski, Jakub & Weron, Rafał, 2016. "On the importance of the long-term seasonal component in day-ahead electricity price forecasting," Energy Economics, Elsevier, vol. 57(C), pages 228-235.
    15. G P Girish & Aviral Kumar Tiwari, 2016. "A comparison of different univariate forecasting models forSpot Electricity Price in India," Economics Bulletin, AccessEcon, vol. 36(2), pages 1039-1057.
    16. Brusaferri, Alessandro & Matteucci, Matteo & Portolani, Pietro & Vitali, Andrea, 2019. "Bayesian deep learning based method for probabilistic forecast of day-ahead electricity prices," Applied Energy, Elsevier, vol. 250(C), pages 1158-1175.
    17. Weron, Rafał & Zator, Michał, 2015. "A note on using the Hodrick–Prescott filter in electricity markets," Energy Economics, Elsevier, vol. 48(C), pages 1-6.
    18. Christian Pape & Arne Vogler & Oliver Woll & Christoph Weber, 2017. "Forecasting the distributions of hourly electricity spot prices," EWL Working Papers 1705, University of Duisburg-Essen, Chair for Management Science and Energy Economics, revised May 2017.
    19. Jakub Nowotarski & Jakub Tomczyk & Rafal Weron, 2013. "Modeling and forecasting of the long-term seasonal component of the EEX and Nord Pool spot prices," HSC Research Reports HSC/13/02, Hugo Steinhaus Center, Wroclaw University of Technology.
    20. Bartosz Uniejewski & Rafał Weron, 2018. "Efficient Forecasting of Electricity Spot Prices with Expert and LASSO Models," Energies, MDPI, vol. 11(8), pages 1-26, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:13:y:2020:i:20:p:5420-:d:430252. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.