IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v392y2025ics0306261925006749.html
   My bibliography  Save this article

Predicting and publishing accurate imbalance prices using Monte Carlo Tree Search

Author

Listed:
  • Pavirani, Fabio
  • Van Gompel, Jonas
  • Karimi Madahi, Seyed Soroush
  • Claessens, Bert
  • Develder, Chris

Abstract

The growing reliance on renewable energy sources, particularly solar and wind, has introduced challenges due to their uncontrollable production. This complicates maintaining the electrical grid balance, prompting some transmission system operators in Western Europe to implement imbalance tariffs that penalize unsustainable power deviations. These tariffs create an implicit demand response framework to mitigate grid instability. Yet, several challenges limit active participation. In Belgium, for example, imbalance prices are only calculated at the end of each 15-minute settlement period, creating high risk due to price uncertainty. This risk is further amplified by the inherent volatility of imbalance prices, discouraging participation. Although transmission system operators provide minute-based price predictions, the system imbalance volatility makes accurate price predictions challenging to obtain and requires sophisticated techniques. Moreover, publishing price estimates can prompt participants to adjust their schedules, potentially affecting the system balance and the final price, adding further complexity. To address these challenges, we propose a Monte Carlo Tree Search method that publishes accurate imbalance prices while accounting for potential response actions. Our approach models the system dynamics using a neural network forecaster and a cluster of virtual batteries controlled by reinforcement learning agents. Compared to Belgium’s current publication method, our technique improves price accuracy by 20.4 % under ideal conditions and by 12.8 % in more realistic scenarios. This research addresses an unexplored, yet crucial problem, positioning this paper as a pioneering work in analyzing the potential of more advanced imbalance price publishing techniques.

Suggested Citation

  • Pavirani, Fabio & Van Gompel, Jonas & Karimi Madahi, Seyed Soroush & Claessens, Bert & Develder, Chris, 2025. "Predicting and publishing accurate imbalance prices using Monte Carlo Tree Search," Applied Energy, Elsevier, vol. 392(C).
  • Handle: RePEc:eee:appene:v:392:y:2025:i:c:s0306261925006749
    DOI: 10.1016/j.apenergy.2025.125944
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261925006749
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.apenergy.2025.125944?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Ajagekar, Akshay & Decardi-Nelson, Benjamin & You, Fengqi, 2024. "Energy management for demand response in networked greenhouses with multi-agent deep reinforcement learning," Applied Energy, Elsevier, vol. 355(C).
    2. Lu, Renzhi & Hong, Seung Ho & Zhang, Xiongfeng, 2018. "A Dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach," Applied Energy, Elsevier, vol. 220(C), pages 220-230.
    3. Eduardo J. Salazar & Mauro Jurado & Mauricio E. Samper, 2023. "Reinforcement Learning-Based Pricing and Incentive Strategy for Demand Response in Smart Grids," Energies, MDPI, vol. 16(3), pages 1-33, February.
    4. Thibaut Th'eate & Antonio Sutera & Damien Ernst, 2023. "Matching of Everyday Power Supply and Demand with Dynamic Pricing: Problem Formalisation and Conceptual Analysis," Papers 2301.11587, arXiv.org.
    5. Rahim, Sahar & Wang, Zhen & Ju, Ping, 2022. "Overview and applications of Robust optimization in the avant-garde energy grid infrastructure: A systematic review," Applied Energy, Elsevier, vol. 319(C).
    6. Vázquez-Canteli, José R. & Nagy, Zoltán, 2019. "Reinforcement learning for demand response: A review of algorithms and modeling techniques," Applied Energy, Elsevier, vol. 235(C), pages 1072-1089.
    7. Shen, Bo & Ghatikar, Girish & Lei, Zeng & Li, Jinkai & Wikler, Greg & Martin, Phil, 2014. "The role of regulatory reforms, market changes, and technology development to make demand response a viable resource in meeting energy challenges," Applied Energy, Elsevier, vol. 130(C), pages 814-823.
    8. Xie, Jiahan & Ajagekar, Akshay & You, Fengqi, 2023. "Multi-Agent attention-based deep reinforcement learning for demand response in grid-responsive buildings," Applied Energy, Elsevier, vol. 342(C).
    9. Micha{l} Narajewski, 2022. "Probabilistic forecasting of German electricity imbalance prices," Papers 2205.11439, arXiv.org.
    10. Julian Schrittwieser & Ioannis Antonoglou & Thomas Hubert & Karen Simonyan & Laurent Sifre & Simon Schmitt & Arthur Guez & Edward Lockhart & Demis Hassabis & Thore Graepel & Timothy Lillicrap & David , 2020. "Mastering Atari, Go, chess and shogi by planning with a learned model," Nature, Nature, vol. 588(7839), pages 604-609, December.
    11. Michał Narajewski, 2022. "Probabilistic Forecasting of German Electricity Imbalance Prices," Energies, MDPI, vol. 15(14), pages 1-17, July.
    12. David Silver & Julian Schrittwieser & Karen Simonyan & Ioannis Antonoglou & Aja Huang & Arthur Guez & Thomas Hubert & Lucas Baker & Matthew Lai & Adrian Bolton & Yutian Chen & Timothy Lillicrap & Fan , 2017. "Mastering the game of Go without human knowledge," Nature, Nature, vol. 550(7676), pages 354-359, October.
    13. Papadis, Elisa & Tsatsaronis, George, 2020. "Challenges in the decarbonization of the energy sector," Energy, Elsevier, vol. 205(C).
    14. Lacal Arantegui, Roberto & Jäger-Waldau, Arnulf, 2018. "Photovoltaics and wind status in the European Union after the Paris Agreement," Renewable and Sustainable Energy Reviews, Elsevier, vol. 81(P2), pages 2460-2471.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Antonopoulos, Ioannis & Robu, Valentin & Couraud, Benoit & Kirli, Desen & Norbu, Sonam & Kiprakis, Aristides & Flynn, David & Elizondo-Gonzalez, Sergio & Wattam, Steve, 2020. "Artificial intelligence and machine learning approaches to energy demand-side response: A systematic review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 130(C).
    2. Perera, A.T.D. & Kamalaruban, Parameswaran, 2021. "Applications of reinforcement learning in energy systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 137(C).
    3. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    4. Muhammad Ikram & Daryoush Habibi & Asma Aziz, 2025. "Networked Multi-Agent Deep Reinforcement Learning Framework for the Provision of Ancillary Services in Hybrid Power Plants," Energies, MDPI, vol. 18(10), pages 1-34, May.
    5. Bashyal, Atit & Boroukhian, Tina & Veerachanchai, Pakin & Naransukh, Myanganbayar & Wicaksono, Hendro, 2025. "Multi-agent deep reinforcement learning based demand response and energy management for heavy industries with discrete manufacturing systems," Applied Energy, Elsevier, vol. 392(C).
    6. Shafqat Jawad & Junyong Liu, 2020. "Electrical Vehicle Charging Services Planning and Operation with Interdependent Power Networks and Transportation Networks: A Review of the Current Scenario and Future Trends," Energies, MDPI, vol. 13(13), pages 1-24, July.
    7. Gokhale, Gargya & Claessens, Bert & Develder, Chris, 2022. "Physics informed neural networks for control oriented thermal modeling of buildings," Applied Energy, Elsevier, vol. 314(C).
    8. Ibrahim, Muhammad Sohail & Dong, Wei & Yang, Qiang, 2020. "Machine learning driven smart electric power systems: Current trends and new perspectives," Applied Energy, Elsevier, vol. 272(C).
    9. Nickelsen, Daniel & Müller, Gernot, 2025. "Bayesian hierarchical probabilistic forecasting of intraday electricity prices," Applied Energy, Elsevier, vol. 380(C).
    10. Savino, Sabrina & Minella, Tommaso & Nagy, Zoltán & Capozzoli, Alfonso, 2025. "A scalable demand-side energy management control strategy for large residential districts based on an attention-driven multi-agent DRL approach," Applied Energy, Elsevier, vol. 393(C).
    11. Li, Wenqing & Ni, Shaoquan, 2022. "Train timetabling with the general learning environment and multi-agent deep reinforcement learning," Transportation Research Part B: Methodological, Elsevier, vol. 157(C), pages 230-251.
    12. Dengiz, Thomas & Jochem, Patrick & Fichtner, Wolf, 2019. "Demand response with heuristic control strategies for modulating heat pumps," Applied Energy, Elsevier, vol. 238(C), pages 1346-1360.
    13. Lee, Hyun-Suk, 2024. "Automated tariff design for energy supply–demand matching based on Bayesian optimization: Technical framework and policy implications," Energy Policy, Elsevier, vol. 188(C).
    14. Xie, Jiahan & Ajagekar, Akshay & You, Fengqi, 2023. "Multi-Agent attention-based deep reinforcement learning for demand response in grid-responsive buildings," Applied Energy, Elsevier, vol. 342(C).
    15. Eduardo J. Salazar & Mauro Jurado & Mauricio E. Samper, 2023. "Reinforcement Learning-Based Pricing and Incentive Strategy for Demand Response in Smart Grids," Energies, MDPI, vol. 16(3), pages 1-33, February.
    16. Ciaran O’Connor & Mohamed Bahloul & Steven Prestwich & Andrea Visentin, 2025. "A Review of Electricity Price Forecasting Models in the Day-Ahead, Intra-Day, and Balancing Markets," Energies, MDPI, vol. 18(12), pages 1-40, June.
    17. Bruneel, Robin & Schuurmans, Mathijs & Patrinos, Panagiotis, 2025. "Optimal intraday power trading for single-price balancing markets: An adaptive risk-averse strategy using mixture models," Applied Energy, Elsevier, vol. 389(C).
    18. Liu, Jiejie & Ma, Yanan & Chen, Ying & Zhao, Chunlu & Meng, Xianyang & Wu, Jiangtao, 2025. "Multi-agent deep reinforcement learning-based cooperative energy management for regional integrated energy system incorporating active demand-side management," Energy, Elsevier, vol. 319(C).
    19. Ahmed Ismail & Mustafa Baysal, 2023. "Dynamic Pricing Based on Demand Response Using Actor–Critic Agent Reinforcement Learning," Energies, MDPI, vol. 16(14), pages 1-19, July.
    20. Svetozarevic, B. & Baumann, C. & Muntwiler, S. & Di Natale, L. & Zeilinger, M.N. & Heer, P., 2022. "Data-driven control of room temperature and bidirectional EV charging using deep reinforcement learning: Simulations and experiments," Applied Energy, Elsevier, vol. 307(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:392:y:2025:i:c:s0306261925006749. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.