Carbon cap based multi-energy sharing among heterogeneous microgrids using multi-agent safe reinforcement learning method with credit assignment and sequential update

Harnessing the demand-side decarbonization potential of multi-energy microgrid (MEMG) communities is crucial for achieving cost-effective emission reduction. To leverage the demand-side multi-energy substitution among heterogeneous MGs for carbon mitigation, a carbon-constrained multi-energy peer-to-peer sharing model is proposed. Clarifying the emission responsibility of energy consumption relies on the carbon emission flow (CEF) model. However, the nonlinear coupling between the CEF and power flow presents obstacles to decentralized decision-making of MGs in real-world applications. For the decomposition, the model with a global carbon cap is formulated as a constrained Markov Game. To remedy the lack of coordination in existing algorithms, this paper proposes a coordinated multi-agent safe reinforcement learning method integrating the emission violation gradient along with the energy profit gradient into the carbon-aware trading policy learning process, thus striking the balance between economic efficiency and decarbonization for each MEMG. Furthermore, a sequential policy update mechanism is introduced to effectively coordinate the heterogeneous MEMGs, distilling each MG's unique contribution to both profit and emission. Numerical results validate that the proposed method outperforms state-of-the-art safe reinforcement learning algorithms and the traditional carbon regulation method, in terms of preserving 96.7 %maximum profitability while complying with stringent carbon constraint. Enables 9.7 % more emission reduction in comparison with generation-side regulation method.