A scenario-based multi-agent reinforcement learning approach for efficient solving to long-term optimization of cascade hydropower reservoirs

The optimal scheduling of hydropower stations plays a critical role in facilitating the low-carbon transition of the energy mix and providing power supply support for new-type power systems. However, uncertainty in long-term streamflow forecasting and the inherent complexity of large-scale cascade reservoir optimization make it challenging to obtain reliable scheduling schemes that balance operational risks and power generation benefits within acceptable time. To address these challenges, an integrated methodology is proposed for cascade reservoir optimization. First, streamflow uncertainty is incorporated via a scenario-based rolling-horizon optimization framework. Second, a spatial-dimensional reinforcement learning modeling approach is introduced to reduce the action space dimensionality. Third, a novel Multi-agent Reinforcement Learning Progressive Optimality Algorithm (MARL-POA) is proposed to coordinate decisions across temporal and spatial dimensions. The research results demonstrate that: (1) Compared with deterministic optimization, the scenario-based computational framework can better consider operation risks and yield more robust scheduling schemes; (2) For typical inflow years, the water level operation process derived via MARL-POA can fully utilize the storage capacity coordination of large hydropower stations; (3) The proposed MARL-POA achieves computationally efficient and high-accuracy solutions for multiple uncertain streamflow scenarios. Accordingly, the proposed approach can provide effective technical support for the practical engineering operation of cascade hydropower systems.