Short-term voltage stability emergency control strategy pre-formulation for massive operating scenarios via adversarial reinforcement learning

The high penetration of renewable energy shifts the randomness and uncertainty of power systems, challenging traditional interpolation-based emergency control strategy pre-formulation. Deep reinforcement learning (DRL)-based approaches provide a promising alternative to tackle this issue. However, the applicability of prevalent DRL-based methods is limited by the safety concerns in low-frequency high-risk conditions and by the computational costs for tackling various fault scenarios. To address these issues, we develop a safe reinforcement learning (SRL)-based emergency control framework against short-term voltage instability. First, considering the need for scanning numerous fault scenarios in large-scale power systems, we employ u-shapelet-based time series clustering to group faults with similar response characteristics, which simplifies the construction of emergency control strategies for various fault scenarios while guaranteeing performance. After clustering, a neural network-based security margin estimator for safety quantification is incorporated with a risky action corrector via the estimated margin’s gradient projection for safety guarantee to form an SRL-enabled decision-making agent, achieving efficient and safe strategy pre-formulation. Further, adversarial sample generation is performed to gather extreme scenarios for the SRL-based agent, improving robustness and applicability. Comprehensive tests on the IEEE 39-bus system and the Guangdong Provincial Power Grid demonstrate the effectiveness of the proposed framework.