UAV charging dock location and patrol path planning under epistemic uncertainty using distributed reinforcement learning

This paper tackles the integrated problem of UAV charging dock placement and patrol-routing under uncertainty. Signal base stations are adopted to provide continuous control links for UAVs, and only tower-based stations can host charging docks. We propose a two-stage framework that combines a Distributed Multi-Agent Reinforcement Learning (DMARL) scheme with a time–space network model. In Stage 1, each tower-based station is represented by an independent agent that learns, with only local observations, whether to install a dock; the collective decisions converge to a stable, scalable deployment plan. In Stage 2, the resulting dock configuration is fed into a time–space network formulation that optimizes UAV travel, patrol and charging schedules, guaranteeing uninterrupted operation. To balance efficiency and equity, we introduce an Adaptive Weighted Round-Robin Strategy (AWRRS) that dynamically adjusts patrol frequencies. Extensive experiments on real city-wide signal-station data show that DMARL attains near-optimal utility without prior information, outperforming centralized RL. Different AWRRS variants demonstrate controllable trade-offs between efficiency and equity, validating the framework’s adaptability to diverse mission requirements.