A stochastic model-free reinforcement learning framework for optimizing runway capacity management under uncertainty

Air traffic operations are often subject to congestion due to rising air travel demand levels and capacity limitations at airport and airspace resources. These capacity constraints are frequently exacerbated by adverse weather conditions, one of the primary causes of flight delays and additional operational costs. To mitigate the impact of demand-capacity imbalances on overall aviation system performance, there is a pressing need for more advanced Air Traffic Flow Management (ATFM) processes, which must be able to better address the complexities and challenges arising from dynamic and stochastic operational environments. In recent years, machine learning techniques have emerged as promising tools to enhance ATFM decision-making, offering potential solutions to these challenges. This study investigates the application of different reinforcement learning (RL) approaches and algorithms for runway capacity management under uncertainty, including both runway configuration selection and airport service rate allocation decisions. The problem is formulated as a Markov Decision Process (MDP), and two approaches are proposed: data-based and forecast-based. Both approaches leverage a state-of-the-art model-free RL method, with the Maskable Proximal Policy Optimization (PPO) algorithm, which is compared to a traditional RL algorithm - Deep Q-Network (DQN). The results reveal that both algorithms perform similarly, with our stochastic forecast-based and incremental data-driven approaches outperforming traditional methods. These approaches offer notable reductions in delay costs compared to the baseline policy typically used in practice and yield results comparable to the best theoretical solutions derived from genetic algorithms. This study highlights two efficient methods for addressing runway capacity management challenges at airports and provides valuable insights into data-driven ATFM optimization and policy implications.