Multi-agent heterogeneous graph reinforcement learning for electric vehicle routing and charging scheduling in coupled power-transportation networks

The global decarbonization agenda facilitates the development of electrifying transportation. With the vehicle-to-grid (V2G) technology and charging navigation strategy, electric vehicles (EVs) are emerging as promising participants in the coupled power and transportation network (CPTN). In recent years, deep reinforcement learning has made significant progress in solving electric vehicle (EV) routing and charging scheduling problems. However, existing methods fail to consider the heterogeneous nature of CPTN, and the potential impact of local marginal price (LMP) has not been fully explored, limiting their effectiveness in real-world applications. To address these limitations, this paper proposes a multi-agent heterogeneous graph reinforcement learning method for EV routing and charging scheduling, aiming to minimize the total operational cost for the CPTN. First, we design a heterogeneous graph attention (HGAT) mechanism that effectively captures the cross-domain relationships between the power distribution network (PDN) and transportation network (TN). Second, we propose an online LMP generation mechanism that captures real-time conditions of the PDN to guide when and where charging activities should occur. Finally, we introduce a multi-agent proximal policy optimization algorithm with a multi-head actor-critic network to optimize the hybrid decision-making process. Extensive experimental results using real-world datasets demonstrate that our proposed method achieves state-of-the-art performance in terms of cost-effectiveness.