Learning in practice: reinforcement learning-based traffic signal control augmented with actuated control

Most Reinforcement Learning (RL) based Traffic Signal Control (TSC) models are trained in simulation platforms, which inevitably suffer from the performance degradation after deployment due to the mismatch between the traffic simulator and the real-world traffic system. Toward the real-world training of TSC agents, this study integrates the actuated control into the RL-based TSC framework as a defense mechanism, enabling the traffic system to remain resilient to suboptimal actions of agent during the real-world training process. The introduction of actuated control can significantly reduce the green time wastes due to unreasonable timing plans. The numerical results demonstrate that the proposed approach can reduce interferences to the traffic system operation during the real-world training of TSC agents, particularly at the early stage of agent learning, thereby promoting the practical deployment of RL-based TSC models. Meanwhile, agents can converge to better signal policies with the help of defense mechanism.