Deep reinforcement learning based hierarchical energy management for virtual power plant with aggregated multiple heterogeneous microgrids

The operational uncertainties for different forms of renewable energy sources (RES) and their high penetration in microgrids (MG) impose challenges to their flexible operation. This paper addresses the cooperation within a virtual power plant (VPP) aggregated with multiple heterogeneous MGs. The VPP, managed by VPP operators, serves as an intermediary entity to facilitate the economic and low-carbon operation of MGs. This paper proposes a deep reinforcement learning (DRL) based collaborative energy management framework consisting of three energy management stages: internal price setting, MG scheduling and VPP's ESS management. Multiple DRL agents are designed for different roles in these three stages, and adversarial training is conducted to address the internal pricing issues. The proposed solution is assessed through extensive simulation experiments with the use of real datasets. The simulation results confirmed that the proposed collaborative management solution can benefit both the VPP operator and MGs in terms of improved profits.