Two-layered optimal scheduling under a semi-model architecture of hydro-wind-solar multi-energy systems with hydrogen storage

With the development of artificial intelligence (AI), it is a research hotspot using model-free deep reinforcement learning (DRL) to improve the intelligent level of hydro-wind-solar-hydrogen systems. However, the extent to which DRL should be applied to achieve maximum economic benefits has not yet been quantified. Additionally, as hydrogen storage station charging and discharging times increase, traditional scheduling strategies face new challenges. In this study, the optimal scheduling is divided into two layers to adapt to the particularity of hydrogen storage system (HSS). The model-free and model-based optimization architecture is established for the upper and lower layers of the scheduling, respectively, called semi-model architecture (SMA). At the upper layer of SMA, the DRL learns environmental information, optimizing energy scheduling by predicting loads under demand response mechanisms and renewable energy output. The lower layer employs model predictive control (MPC) to optimize energy storage, using a 1-h scheduling time scale and a 4-h rolling optimization scale for HSS. The simulations verify that the 4-h rolling optimization we proposed brings shorter simulation times and higher economic benefits compared to other time scales. The SMA-based scheduling improves economic efficiency by 40 % and 23 % compared to model-free DRL-based scheduling using DoubleDQN and SARSA algorithms, respectively.