Long-term efficient energy management for multi-station collaborative electric vehicle charging: A transformer-based multi-agent reinforcement learning approach

With the rapid adoption and increasing sales of electric vehicles, energy management for electric vehicle charging and maximizing the utilization of green energy have become increasingly critical. Existing studies have demonstrated that reinforcement learning plays a key role in enhancing power dispatch efficiency. However, in complex scenarios involving multiple stations and charging sites, significant challenges remain in leveraging mutual information to capture long-term temporal relationships and addressing the massive state and action spaces. To fill the research gaps in large-scale data, significant long-term temporal dependencies, and communication challenges in multi-station collaborative electric vehicle charging energy management, we propose a transformer-based multi-agent reinforcement learning algorithm, MAHEM. This algorithm leverages the popular transformer model to capture long-term temporal features in sequential data during distributed execution. By utilizing the transformer architecture, our method reduces the complexity of the action space through Q-value decomposition. Different agents effectively communicate through the attention mechanism, while the transformer model efficiently captures long-term temporal information, accelerating training and convergence by predicting future states. Experimental results show that, compared to existing baselines, our method reduces the total charging cost across stations by 31.6 % and achieves optimal performance across various environments, robustness tests, and transfer tests. This highlights the practicality and effectiveness of MAHEM in addressing the challenges of EV energy management systems.