Multi-agent deep reinforcement learning based low-carbon Economy energy planning strategy in IES connected with microgrid

Utilizing multiple renewable energy sources (RESs) in an integrated energy system (IES) is crucial for reducing electricity costs and carbon emissions. However, the high costs of energy storage system (ESS) and the fluctuation of renewable energy challenge the performance of IESs. Utilizing the renewable energy system and compensatory electricity generation system, this paper proposes a deep reinforcement learning (DRL) based energy planning strategy (EPS) to address the challenges such as high costs of ESS and carbon emission tax and the operational security of internal power grid, which can support energy for remote industrial areas. Integrating ESS and combined heat and power system (CHP), a carbon-energy price consumption model in IES is built based on the energy flow in gas, thermal, and power systems. A two-step robust optimization model is proposed to accurately capture the worst cases of interaction characteristics between the IES and the RESs. This model is coupled with the Markov decision process (MDP) to propose a two-layer MDP-based two-phase Nash optimization model. Moreover, an optimized differential evolution mutation (DEM) minimax multi-agent DRL-based EPS in IES is proposed to solve the EPS between every subsystem. The effectiveness of the proposed algorithm is validated through numerical simulation results.