Integrating peer-to-peer electricity trading and Var support service within distribution networks: An evolutionary multi-agent deep reinforcement learning approach

This paper proposes a novel multi-market framework for peer-to-peer (P2P) electricity trading and Var support service within the distribution network. The proposed framework is designed to harness both active and reactive power flexibilities from prosumers to achieve mutual benefits for both market participants and DSO. To maximize the flexibility of distributed energy resources (DERs) and enhance the hosting capacity of the distribution network, the proposed framework leverages dynamic operating envelopes (DOEs) as security constraints. To ensure fair profit allocation and foster DSO-prosumer coordination, a pricing mechanism for Var support service is developed based on loss sensitivity factors. This mechanism provides explicit incentives for prosumers equipped with smart inverters to contribute to network loss reduction through Var support service. To address the decision-making complexities inherent in such a multi-market environment, an evolutionary algorithm-assisted multi-agent twin delayed deep deterministic policy gradient (EA-MATD3) algorithm is proposed. Case studies conducted on the IEEE-33 bus system verify that the multi-market framework significantly lowers prosumers' cost while effectively reducing total network loss. Also, compared with other multi-agent deep reinforcement learning algorithms, the proposed EA-MATD3 algorithm demonstrates superior exploration capability. Furthermore, the scalability and generalizability of the EA-MATD3 algorithm are validated using a large-scale real-world Swiss-161 bus distribution network. Numerical results also highlight an interdependence between active power trading and Var support service, suggesting that multi-market framework is essential for distribution network.