Game theoretic Markov decision process for service scheduling in stochastic railway-based multi-modal transit network

This paper presents a game theoretic Markov decision process for urban transit service scheduling in a stochastic multi-modal network. The Markov decision game consists of multiple decision-making agents, in which we first have an agent that manipulates the number of connecting bus services and hence passenger influx to a core rail transit network. This agent operates as an attacker to construct a worst-case scenario for deriving the core train operational plans under a robust optimization framework. In response, the rail transit agents would defend these disturbances from the connecting mode by managing the corresponding transit service headways, routes to serve, and fleet sizes of vehicles to deploy. The objective of the transit operation is to minimize the passenger waiting times due to disturbances as well as the associated operating costs. To address the inherent computational complexity, we implement an adversarial multi-agent deep reinforcement learning framework in which the state and decision spaces of each agent involved are approximated by artificial neural network surrogates with a decentralized policy function and a centralized value function approximation. The reinforcement learning framework is to be trained through a multi-agent proximal policy optimization algorithm in a stochastic environment. The proposed framework is tested using a real-world multi-modal network with operational data from the Hong Kong Light Rail Transit and the connecting bus network. Experimental results show that the computing framework trained with the proximal policy optimization process outperforms various well-established offline meta-heuristic approaches in solution quality and computational effectiveness. Additional experiments also demonstrate the robustness and adaptability of the proposed attacker-defender model under different scenarios. This study contributes to adaptive and robust multi-modal transit services with advanced control and optimization technologies.