Distributed hierarchical reinforcement learning for dynamic maintenance scheduling of large-scale airline fleets

Decision-making for modern airline maintenance requires life-cycle evaluation across aircraft, components, and bases, with hierarchical interactions among routing, shop capacity, and turnaround. Large-scale fleet modeling is challenging due to the heterogeneity of fleets, multi-airport logistics, degradation dynamics, tight resource coupling, and partial observability. Dynamic maintenance scheduling based on standard reinforcement learning (RL) is often unstable and slow to converge, particularly for long-horizon, multi-level scheduling. This paper proposes a Hierarchical Multi-Agent Distributed Reinforcement Learning (HMADRL) framework for dynamic maintenance scheduling of large-scale aviation fleets. A parameterized environment is introduced to capture multi-aircraft, multi-airport, multi-task operations with degradation and resource coupling. To address the challenges of multi-agent coordination and multi-timescale decision-making, the proposed HMADRL framework adopts a two-level policy structure. Additionally, the algorithm integrates shared communication protocols and coordinated reward structures to enhance learning efficiency and agent cooperation. Numerical evaluations on small (10 aircraft, 2 airports) and large (200 aircraft, 4 airports) settings benchmark HMADRL against baseline algorithms. HMADRL shows a reduction of about 23 % to 50 % in our simulations, converges faster and more stably, and increases task completion. Ablation results indicate that both best-learner synchronization and auxiliary returns are essential to the observed gains.