Fully autonomous load frequency control for integrated energy system with massive energy prosumers using multi-agent deep meta reinforcement learning

In Interconnected Integrated Energy Systems (IIES), grid operators face the challenge of dealing with intermittent and stochastic disturbances caused by energy prosumers, while considering the multi-energy constraints and exploiting the fast response capability of prosumers to optimize the benefits of both regulation service providers and the grid. To address these challenges, this paper proposes a Fully Autonomous Load Frequency Control (FA-LFC) approach for IIES in performance-based frequency regulation markets. This approach considers each regulation service provider as an independent decision-making agent that can autonomously adjust its policy based on the state of its area and the global objective. The agents use a novel reinforcement learning algorithm called Maximum Entropy Multi-Agent Deep Meta Actor Critic (MEMA-DMAC), which combines meta-learning and multi-agent learning with a maximum entropy exploration policy. The MEMA-DMAC algorithm enables the agents to learn from high-value demonstrations of different regulation tasks, as well as to account for the multi-energy constraints through centralized learning. The proposed method is validated on a four-area LFC model of China Southern Grid (CSG).