Deep Reinforcement Q-Learning for Intelligent Traffic Control in Mass Transit

Traffic control in mass transit consists of the regulation of both vehicle dynamics and passenger flows. While most of the existing approaches focus on the optimization of vehicle dwell time, vehicle time headway, and passenger stocks, we propose in this article an approach which also includes the optimization of the passenger inflows to the platforms. We developed in this work a deep reinforcement Q-learning model for the traffic control in a mass transit line. We first propose a new mathematical traffic model for the train and passengers dynamics. The model combines a discrete-event description of the vehicle dynamics, with a macroscopic model for the passenger flows. We use this new model as the environment of the traffic in mass transit for the reinforcement learning optimization. For this aim, we defined, under the new traffic model, the state variables as well as the control ones, including in particular the number of running vehicles, the vehicle dwell times at stations, and the passenger inflow to platforms. Second, we present our new deep Q-network (DQN) model for the reinforcement learning (RL) with the state representation, action space, and reward function definitions. We also provide the neural network architecture as well as the main hyper-parameters. Finally, we give an evaluation of the model under multiple scenarios. We show in particular the efficiency of the control of the passenger inflows into the platforms.