Optimizing battery-swapping systems management for electric micro-mobility: A reinforcement learning approach

As electric micro-mobility vehicles (EMVs) such as e-bikes and e-scooters increasingly meet daily commuting and delivery needs, battery swapping services have become a convenient charging method. However, unordered battery swapping disrupts user experience and operational efficiency at battery swapping stations (BSSs). Additionally, the mismatch between battery-swapping demand and available batteries leads to idle batteries and resource waste. Therefore, it is crucial to provide organized battery-swapping recommendation services while also channeling stored energy from idle batteries back to the grid. We model the operation of EMVs and battery-swapping stations (BSSs) as a Partially Observable Markov Decision Process (POMDP), where each BSS acts as an agent interacting with the environment and other stations. We introduce a multi-agent hierarchical reinforcement learning model to address the asynchronous nature of swapping recommendations and charging-discharging actions. Our model aims to minimize both the total swapping time and the operational costs of BSSs. Through numerical studies using EMV battery-swapping demand data from Nanjing, we demonstrate the model’s effectiveness in reducing detour and queuing times, improving swapping success rates, and balancing service loads. The proposed model and algorithm show strong generalization capabilities across different operational environments, indicating their potential for broader application in optimizing EMV BSS management.