IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v422y2026ics0306261926009232.html

Optimal scheduling of multi-agent hydrogen-integrated energy systems based on improved spatio-temporal clustering and multi-agent optimal compromise reinforcement learning

Author

Listed:
  • Wang, Ziren
  • Zhang, Yanchi
  • Hu, Jian
  • Zhang, Yuyuan
  • Li, Wei
  • Zhang, Qin

Abstract

While hydrogen-integrated energy systems (HIES) are crucial for the green transition, existing optimal scheduling models predominantly treat hydrogen merely as a passive conversion medium rather than an independent decision-making entity. Furthermore, current models struggle to manage the severe action conflicts and extreme tail-risks induced by spatiotemporal uncertainties in multi-agent environments. To address these critical knowledge gaps, this paper proposes a novel optimal scheduling framework that integrates improved spatiotemporal clustering with multi-agent optimal compromise reinforcement learning (MAOCPPO). First, considering the uncertainties of wind and solar power outputs alongside their temporal correlations with electrical loads, a multiple operational scenario extraction model based on two-stage spatiotemporal clustering is established. Second, based on the extracted probabilistic operational scenarios, Conditional Value-at-Risk (CVaR) is applied for the measurement of the risks induced by uncertainties. Also, an optimal scheduling model for the multi-agent HIES is formulated for the minimization of the total cost. Third, a multi-agent model is constructed for each participating entity, and the proposed optimal scheduling model is solved within a multi-agent reinforcement learning (MARL) framework. To address the issues of insufficient collaboration, action conflicts, and low optimization efficiency inherent in the traditional Multi-Agent Proximal Policy Optimization (MAPPO) algorithm—which arise from the strict independence of states and actions—this paper enhances state-space collaboration by incorporating the action information of adjacent agents. Furthermore, integrating the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS), a multi-scheme evaluation mechanism is constructed to screen for the action combination that represents the optimal compromise. This yields the multi-agent optimal compromise reinforcement learning algorithm, which significantly improves both inter-agent collaboration and solving efficiency. Finally, case study simulations demonstrate the superior solution accuracy and optimization performance of the proposed model and algorithm.

Suggested Citation

  • Wang, Ziren & Zhang, Yanchi & Hu, Jian & Zhang, Yuyuan & Li, Wei & Zhang, Qin, 2026. "Optimal scheduling of multi-agent hydrogen-integrated energy systems based on improved spatio-temporal clustering and multi-agent optimal compromise reinforcement learning," Applied Energy, Elsevier, vol. 422(C).
  • Handle: RePEc:eee:appene:v:422:y:2026:i:c:s0306261926009232
    DOI: 10.1016/j.apenergy.2026.128269
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261926009232
    Download Restriction: Full text may be restricted for ScienceDirect subscribers

    File URL: https://libkey.io/10.1016/j.apenergy.2026.128269?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:422:y:2026:i:c:s0306261926009232. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.