IDEAS home Printed from https://ideas.repec.org/p/syd/wpaper/2023-03.html
   My bibliography  Save this paper

Machine Learning and Deep Learning Forecasts of Electricity Imbalance Prices

Author

Listed:
  • Sinan Deng
  • John Inekwe
  • Vladimir Smirnov
  • Andrew Wait
  • Chao Wang

Abstract

In this paper, we propose a seasonal attention mechanism, the effectiveness of which is evaluated via the Bidirectional Long Short-Term Memory (BiLSTM) model. We compare its performance with alternative deep learning and machine learning models in forecasting the balancing settlement prices in the electricity market of Great Britain. Critically, the Seasonal Attention-Based BiLSTM framework provides a superior forecast of extreme prices with an out-of-sample gain in the predictability of 25-37% compared with models in the literature. Our forecasting techniques could aid both market participants, to better manage their risk and assign their assets, and policy makers, to operate the system at lower cost.

Suggested Citation

  • Sinan Deng & John Inekwe & Vladimir Smirnov & Andrew Wait & Chao Wang, 2023. "Machine Learning and Deep Learning Forecasts of Electricity Imbalance Prices," Working Papers 2023-03, University of Sydney, School of Economics.
    • Handle: RePEc:syd:wpaper:2023-03
    as

    Download full text from publisher

    File URL: http://econ-wpseries.com/2023/202303.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Derek W. Bunn & Angelica Gianfreda & Stefan Kermer, 2018. "A Trading-Based Evaluation of Density Forecasts in a Real-Time Electricity Market," Energies, MDPI, vol. 11(10), pages 1-13, October.
    2. Goodarzi, Shadi & Perera, H. Niles & Bunn, Derek, 2019. "The impact of renewable energy forecast errors on imbalance volumes and electricity spot prices," Energy Policy, Elsevier, vol. 134(C).
    3. Friedman, Jerome H., 2002. "Stochastic gradient boosting," Computational Statistics & Data Analysis, Elsevier, vol. 38(4), pages 367-378, February.
    4. Poplavskaya, Ksenia & de Vries, Laurens, 2019. "Distributed energy resources and the organized balancing market: A symbiosis yet? Case of three European balancing markets," Energy Policy, Elsevier, vol. 126(C), pages 264-276.
    5. Wu, Zhaoyuan & Zhou, Ming & Zhang, Ting & Li, Gengyin & Zhang, Yan & Liu, Xiaojuan, 2020. "Imbalance settlement evaluation for China's balancing market design via an agent-based model with a multiple criteria decision analysis method," Energy Policy, Elsevier, vol. 139(C).
    6. Nikos S. Thomaidis & Georgios D. Dounias, 2012. "A comparison of statistical tests for the adequacy of a neural network regression model," Quantitative Finance, Taylor & Francis Journals, vol. 12(3), pages 437-449, October.
    7. Alexandre Lucas & Konstantinos Pegios & Evangelos Kotsakis & Dan Clarke, 2020. "Price Forecasting for the Balancing Energy Market Using Machine-Learning Regression," Energies, MDPI, vol. 13(20), pages 1-16, October.
    8. Derek W. Bunn and Stefan O.E. Kermer, 2021. "Statistical Arbitrage and Information Flow in an Electricity Balancing Market," The Energy Journal, International Association for Energy Economics, vol. 0(Number 5).
    9. Kristiansen, Tarjei, 2007. "The Nordic approach to market-based provision of ancillary services," Energy Policy, Elsevier, vol. 35(7), pages 3681-3700, July.
    10. Francesco Lisi and Enrico Edoli, 2018. "Analyzing and Forecasting Zonal Imbalance Signs in the Italian Electricity Market," The Energy Journal, International Association for Energy Economics, vol. 0(Number 5).
    11. Vandezande, Leen & Meeus, Leonardo & Belmans, Ronnie & Saguan, Marcelo & Glachant, Jean-Michel, 2010. "Well-functioning balancing markets: A prerequisite for wind power integration," Energy Policy, Elsevier, vol. 38(7), pages 3146-3154, July.
    12. Bueno-Lorenzo, Miriam & Moreno, M. Ángeles & Usaola, Julio, 2013. "Analysis of the imbalance price scheme in the Spanish electricity market: A wind power test case," Energy Policy, Elsevier, vol. 62(C), pages 1010-1019.
    13. Taylor, James W., 2006. "Density forecasting for the efficient balancing of the generation and consumption of electricity," International Journal of Forecasting, Elsevier, vol. 22(4), pages 707-724.
    14. Mureddu, Mario & Meyer-Ortmanns, Hildegard, 2018. "Extreme prices in electricity balancing markets from an approach of statistical physics," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 490(C), pages 1324-1334.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gianfreda, Angelica & Ravazzolo, Francesco & Rossini, Luca, 2020. "Comparing the forecasting performances of linear models for electricity prices with high RES penetration," International Journal of Forecasting, Elsevier, vol. 36(3), pages 974-986.
    2. Micha{l} Narajewski, 2022. "Probabilistic forecasting of German electricity imbalance prices," Papers 2205.11439, arXiv.org.
    3. Kaneko, Nanae & Fujimoto, Yu & Hayashi, Yasuhiro, 2022. "Sensitivity analysis of factors relevant to extreme imbalance between procurement plans and actual demand: Case study of the Japanese electricity market," Applied Energy, Elsevier, vol. 313(C).
    4. Sirin, Selahattin Murat & Yilmaz, Berna N., 2021. "The impact of variable renewable energy technologies on electricity markets: An analysis of the Turkish balancing market," Energy Policy, Elsevier, vol. 151(C).
    5. Michał Narajewski, 2022. "Probabilistic Forecasting of German Electricity Imbalance Prices," Energies, MDPI, vol. 15(14), pages 1-17, July.
    6. Billé, Anna Gloria & Gianfreda, Angelica & Del Grosso, Filippo & Ravazzolo, Francesco, 2023. "Forecasting electricity prices with expert, linear, and nonlinear models," International Journal of Forecasting, Elsevier, vol. 39(2), pages 570-586.
    7. Batalla-Bejerano, Joan & Costa-Campi, Maria Teresa & Trujillo-Baute, Elisa, 2016. "Collateral effects of liberalisation: Metering, losses, load profiles and cost settlement in Spain’s electricity system," Energy Policy, Elsevier, vol. 94(C), pages 421-431.
    8. Brijs, Tom & De Jonghe, Cedric & Hobbs, Benjamin F. & Belmans, Ronnie, 2017. "Interactions between the design of short-term electricity markets in the CWE region and power system flexibility," Applied Energy, Elsevier, vol. 195(C), pages 36-51.
    9. Lago, Jesus & Poplavskaya, Ksenia & Suryanarayana, Gowri & De Schutter, Bart, 2021. "A market framework for grid balancing support through imbalances trading," Renewable and Sustainable Energy Reviews, Elsevier, vol. 137(C).
    10. Russo, Marianna & Kraft, Emil & Bertsch, Valentin & Keles, Dogan, 2022. "Short-term risk management of electricity retailers under rising shares of decentralized solar generation," Energy Economics, Elsevier, vol. 109(C).
    11. Kumar, T. Bharath & Singh, Anoop, 2021. "Ancillary services in the Indian power sector – A look at recent developments and prospects," Energy Policy, Elsevier, vol. 149(C).
    12. Hirth, Lion & Ziegenhagen, Inka, 2013. "Control Power and Variable Renewables A Glimpse at German Data," Energy: Resources and Markets 151373, Fondazione Eni Enrico Mattei (FEEM).
    13. Maciejowska, Katarzyna & Nitka, Weronika & Weron, Tomasz, 2021. "Enhancing load, wind and solar generation for day-ahead forecasting of electricity prices," Energy Economics, Elsevier, vol. 99(C).
    14. Prakash, Abhijith & Bruce, Anna & MacGill, Iain, 2022. "Insights on designing effective and efficient frequency control arrangements from the Australian National Electricity Market," Renewable and Sustainable Energy Reviews, Elsevier, vol. 161(C).
    15. Heeseung Moon & Dongsu Lee & Jeongmin Han & Yongtae Yoon & Seungwan Kim, 2021. "Impact of Imbalance Pricing on Variable Renewable Energies with Different Prediction Accuracies: A Korean Case," Energies, MDPI, vol. 14(13), pages 1-19, July.
    16. Frade, Pedro M.S. & Pereira, João Pedro & Santana, J.J.E. & Catalão, J.P.S., 2019. "Wind balancing costs in a power system with high wind penetration – Evidence from Portugal," Energy Policy, Elsevier, vol. 132(C), pages 702-713.
    17. Joan Batalla-Bejerano & Elisa Trujillo-Baute, 2015. "Analysing the sensitivity of electricity system operational costs to deviations in supply and demand," Working Papers 2015/8, Institut d'Economia de Barcelona (IEB).
    18. Poplavskaya, Ksenia & Lago, Jesus & Strömer, Stefan & de Vries, Laurens, 2021. "Making the most of short-term flexibility in the balancing market: Opportunities and challenges of voluntary bids in the new balancing market design," Energy Policy, Elsevier, vol. 158(C).
    19. Ronald Huisman & Elisa Trujillo-Baute, 2014. "Costs of power supply flexibility: the indirect impact of a Spanish policy change," Working Papers 2014/34, Institut d'Economia de Barcelona (IEB).
    20. Oprea, Simona-Vasilica & Bâra, Adela & Ciurea, Cristian-Eugen, 2022. "A novel cost-revenue allocation computation for the competitiveness of balancing responsible parties, including RES. Insights from the electricity market," Renewable Energy, Elsevier, vol. 199(C), pages 881-894.

    More about this item

    Keywords

    forecasting; electricity; balance settlement prices; Long Short-Term Memory; machine learning.;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:syd:wpaper:2023-03. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Vanessa Holcombe (email available below). General contact details of provider: https://edirc.repec.org/data/deusyau.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.