Domain knowledge embedded constrained reinforcement learning for scheduling of renewable energy power system

With growing renewable penetration, uncertainty-aware day-ahead scheduling of large power system is facing great computational challenges. To address this, a domain knowledge embedded constrained reinforcement learning (DK-CRL) algorithm is proposed. Firstly, a novel constrained Markov decision process (CMDP) of the scheduling problem under renewable uncertainties is constructed, which distinguishes network-level and equipment-level parameters to characterize the respective system states and induces a constrained policy to improve action safety. Secondly, a Lagrangian deep deterministic policy gradient (Lagrangian-DDPG) is adapted to solve the CMDP, where the constraint violations are calculated by the auxiliary cost function. Thirdly, domain knowledge is further incorporated to enhance the performance of Lagrangian-DDPG including the physical knowledge embedded encoder (PKEN) and expert knowledge dataset. PKEN is specially designed to learn the physical characteristics through power flow equations guided graph convolutional network and graph attention network, which is conducive to processing power system state features and solution quality. An expert knowledge dataset is built for the pre-training of the agent based on simplified historical cases, which can reduce unnecessary initial exploration and improve training efficiency. Finally, the effectiveness and applicability of DK-CRL are comprehensively verified on different IEEE cases with 20% renewables.