Reinforcement learning for frequency regulation control and simulation strategy optimization of pumped storage units

The growing reliance on renewable energy in power grids introduces significant challenges to frequency stability due to its intermittent and unpredictable nature. Pumped storage units are vital for mitigating these issues, but traditional control methods often fail to adapt to dynamic grid conditions. This study introduces a reinforcement learning (RL)-based control approach to optimize frequency regulation in pumped storage units. We developed a multi-agent deep RL framework using the Deep Deterministic Policy Gradient algorithm with the Adam optimizer, integrating operational characteristics and dynamic interactions. Multi-source data fusion enhances the robustness of control decisions across diverse conditions. Experimental results show that our method reduces maximum frequency deviation by 40.0%–44.4%, mean frequency deviation by 33.3%–45.5%, and frequency recovery time by 33.1%–42.0% compared to traditional proportional-integral-derivative control in various operating scenarios. Frequency fluctuation count is halved, and the method achieves convergence in eight iterations with a computation time of 0.100 s, outperforming genetic algorithms and particle swarm optimization. This RL-based strategy significantly improves frequency stability, response speed, and energy efficiency, providing a scalable solution for renewable-heavy power systems.