IDEAS home Printed from https://ideas.repec.org/a/eee/transe/v194y2025ics1366554524004915.html
   My bibliography  Save this article

A multi-task deep reinforcement learning approach to real-time railway train rescheduling

Author

Listed:
  • Tang, Tao
  • Chai, Simin
  • Wu, Wei
  • Yin, Jiateng
  • D’Ariano, Andrea

Abstract

In high-speed railway systems, unexpected disruptions can result in delays of trains, significantly affecting the quality of service for passengers. Train Timetable Rescheduling (TTR) is a crucial task in the daily operation of high-speed railways to maintain punctuality and efficiency in the face of such unforeseen disruptions. Most existing studies on TTR are based on integer programming (IP) techniques and are required to solve IP models repetitively in case of disruptions, which however may be very time-consuming and greatly limit their usefulness in practice. Our study first proposes a multi-task deep reinforcement learning (MDRL) approach for TTR. Our MDRL is constructed and trained offline with a large number of historical disruptive events, enabling to generate TTR decisions in real-time for different disruption cases. Specifically, we transform the TTR problem into a Markov decision process considering the retiming and rerouting of trains. Then, we construct the MDRL framework with the definition of state, action, transition, reward, and value function approximations with neural networks for each agent (i.e., rail train), by considering the information of different disruption events as tasks. To overcome the low training efficiency and huge memory usage in the training of MDRL, given a large number of disruptive events in the historical data, we develop a new and high-efficient training method based on a Quadratic assignment programming (QAP) model and a Frank-Wolfe-based algorithm. Our QAP model optimizes only a small number but most “representative” tasks from the historical data, while the Frank-Wolfe-based algorithm approximates the nonlinear terms in the value function of MDRL and updates the model parameters among different training tasks concurrently. Finally, based on the real-world data from the Beijing–Zhangjiakou high-speed railway systems, we evaluate the performance of our MDRL approach by benchmarking it against state-of-the-art approaches in the literature. Our computational results demonstrate that an offline-trained MDRL is able to generate near-optimal TTR solutions in real-time against different disruption scenarios, and it evidently outperforms state-of-art models regarding solution quality and computational time.

Suggested Citation

  • Tang, Tao & Chai, Simin & Wu, Wei & Yin, Jiateng & D’Ariano, Andrea, 2025. "A multi-task deep reinforcement learning approach to real-time railway train rescheduling," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 194(C).
  • Handle: RePEc:eee:transe:v:194:y:2025:i:c:s1366554524004915
    DOI: 10.1016/j.tre.2024.103900
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1366554524004915
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.tre.2024.103900?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Zhan, Shuguang & Kroon, Leo G. & Zhao, Jun & Peng, Qiyuan, 2016. "A rolling horizon approach to the high speed train rescheduling problem in case of a partial segment blockage," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 95(C), pages 32-61.
    2. Chai, Simin & Yin, Jiateng & D’Ariano, Andrea & Liu, Ronghui & Yang, Lixing & Tang, Tao, 2024. "A branch-and-cut algorithm for scheduling train platoons in urban rail networks," Transportation Research Part B: Methodological, Elsevier, vol. 181(C).
    3. Corman, Francesco & D'Ariano, Andrea & Pacciarelli, Dario & Pranzo, Marco, 2010. "A tabu search algorithm for rerouting trains during rail operations," Transportation Research Part B: Methodological, Elsevier, vol. 44(1), pages 175-192, January.
    4. Ying, Cheng-shuo & Chow, Andy H.F. & Chin, Kwai-Sang, 2020. "An actor-critic deep reinforcement learning approach for metro train scheduling with rolling stock circulation under stochastic demand," Transportation Research Part B: Methodological, Elsevier, vol. 140(C), pages 210-235.
    5. Jiateng Yin & Lixing Yang & Andrea D’Ariano & Tao Tang & Ziyou Gao, 2022. "Integrated Backup Rolling Stock Allocation and Timetable Rescheduling with Uncertain Time-Variant Passenger Demand Under Disruptive Events," INFORMS Journal on Computing, INFORMS, vol. 34(6), pages 3234-3258, November.
    6. Ying, Cheng-shuo & Chow, Andy H.F. & Nguyen, Hoa T.M. & Chin, Kwai-Sang, 2022. "Multi-agent deep reinforcement learning for adaptive coordinated metro service operations with flexible train composition," Transportation Research Part B: Methodological, Elsevier, vol. 161(C), pages 36-59.
    7. Zhang, Chuntian & Gao, Yuan & Yang, Lixing & Gao, Ziyou & Qi, Jianguo, 2020. "Joint optimization of train scheduling and maintenance planning in a railway network: A heuristic algorithm using Lagrangian relaxation," Transportation Research Part B: Methodological, Elsevier, vol. 134(C), pages 64-92.
    8. Corman, F. & D’Ariano, A. & Pacciarelli, D. & Pranzo, M., 2012. "Optimal inter-area coordination of train rescheduling decisions," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 48(1), pages 71-88.
    9. Yin, Jiateng & Tang, Tao & Yang, Lixing & Gao, Ziyou & Ran, Bin, 2016. "Energy-efficient metro train rescheduling with uncertain time-variant passenger demands: An approximate dynamic programming approach," Transportation Research Part B: Methodological, Elsevier, vol. 91(C), pages 178-210.
    10. Zhang, Yuchang & Bai, Ruibin & Qu, Rong & Tu, Chaofan & Jin, Jiahuan, 2022. "A deep reinforcement learning based hyper-heuristic for combinatorial optimisation with uncertainties," European Journal of Operational Research, Elsevier, vol. 300(2), pages 418-427.
    11. Julian Schrittwieser & Ioannis Antonoglou & Thomas Hubert & Karen Simonyan & Laurent Sifre & Simon Schmitt & Arthur Guez & Edward Lockhart & Demis Hassabis & Thore Graepel & Timothy Lillicrap & David , 2020. "Mastering Atari, Go, chess and shogi by planning with a learned model," Nature, Nature, vol. 588(7839), pages 604-609, December.
    12. Zhu, Yongqiu & Goverde, Rob M.P., 2019. "Railway timetable rescheduling with flexible stopping and flexible short-turning during disruptions," Transportation Research Part B: Methodological, Elsevier, vol. 123(C), pages 149-181.
    13. D'Ariano, Andrea & Pacciarelli, Dario & Pranzo, Marco, 2007. "A branch and bound algorithm for scheduling trains in a railway network," European Journal of Operational Research, Elsevier, vol. 183(2), pages 643-657, December.
    14. Zhu, Yongqiu & Goverde, Rob M.P., 2020. "Integrated timetable rescheduling and passenger reassignment during railway disruptions," Transportation Research Part B: Methodological, Elsevier, vol. 140(C), pages 282-314.
    15. Andrea D'Ariano & Francesco Corman & Dario Pacciarelli & Marco Pranzo, 2008. "Reordering and Local Rerouting Strategies to Manage Train Traffic in Real Time," Transportation Science, INFORMS, vol. 42(4), pages 405-419, November.
    16. Wei, Tangjian & Batley, Richard & Liu, Ronghui & Xu, Guangming & Tang, Yili, 2024. "A method of time-varying demand distribution estimation for high-speed railway networks with user equilibrium model," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 189(C).
    17. Yan, Yimo & Chow, Andy H.F. & Ho, Chin Pang & Kuo, Yong-Hong & Wu, Qihao & Ying, Chengshuo, 2022. "Reinforcement learning for logistics and supply chain management: Methodologies, state of the art, and future opportunities," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 162(C).
    18. Zhan, Shuguang & Xie, Jiemin & Wong, S.C. & Zhu, Yongqiu & Corman, Francesco, 2024. "Handling uncertainty in train timetable rescheduling: A review of the literature and future research directions," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 183(C).
    19. Šemrov, D. & Marsetič, R. & Žura, M. & Todorovski, L. & Srdic, A., 2016. "Reinforcement learning approach for train rescheduling on a single-track railway," Transportation Research Part B: Methodological, Elsevier, vol. 86(C), pages 250-267.
    20. Zhan, Shuguang & Kroon, Leo G. & Veelenturf, Lucas P. & Wagenaar, Joris C., 2015. "Real-time high-speed train rescheduling in case of a complete blockage," Transportation Research Part B: Methodological, Elsevier, vol. 78(C), pages 182-201.
    21. Zhang, Chuntian & Gao, Yuan & Cacchiani, Valentina & Yang, Lixing & Gao, Ziyou, 2023. "Train rescheduling for large-scale disruptions in a large-scale railway network," Transportation Research Part B: Methodological, Elsevier, vol. 174(C).
    22. Leonardo Lamorgese & Carlo Mannino, 2019. "A Noncompact Formulation for Job-Shop Scheduling Problems in Traffic Management," Operations Research, INFORMS, vol. 67(6), pages 1586-1609, November.
    23. Zhan, Shuguang & Wang, Pengling & Wong, S.C. & Lo, S.M., 2022. "Energy-efficient high-speed train rescheduling during a major disruption," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 157(C).
    24. Li, Wenqing & Ni, Shaoquan, 2022. "Train timetabling with the general learning environment and multi-agent deep reinforcement learning," Transportation Research Part B: Methodological, Elsevier, vol. 157(C), pages 230-251.
    25. Wang, Xuekai & D’Ariano, Andrea & Su, Shuai & Tang, Tao, 2023. "Cooperative train control during the power supply shortage in metro system: A multi-agent reinforcement learning approach," Transportation Research Part B: Methodological, Elsevier, vol. 170(C), pages 244-278.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ying, Chengshuo & Chow, Andy H.F. & Yan, Yimo & Kuo, Yong-Hong & Wang, Shouyang, 2024. "Adaptive rescheduling of rail transit services with short-turnings under disruptions via a multi-agent deep reinforcement learning approach," Transportation Research Part B: Methodological, Elsevier, vol. 188(C).
    2. Zhang, Chuntian & Gao, Yuan & Cacchiani, Valentina & Yang, Lixing & Gao, Ziyou, 2023. "Train rescheduling for large-scale disruptions in a large-scale railway network," Transportation Research Part B: Methodological, Elsevier, vol. 174(C).
    3. Zhan, Shuguang & Xie, Jiemin & Wong, S.C. & Zhu, Yongqiu & Corman, Francesco, 2024. "Handling uncertainty in train timetable rescheduling: A review of the literature and future research directions," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 183(C).
    4. Wang, Xuekai & D’Ariano, Andrea & Su, Shuai & Tang, Tao, 2023. "Cooperative train control during the power supply shortage in metro system: A multi-agent reinforcement learning approach," Transportation Research Part B: Methodological, Elsevier, vol. 170(C), pages 244-278.
    5. Chang Han & Leishan Zhou & Bin Guo & Yixiang Yue & Wenqiang Zhao & Zeyu Wang & Hanxiao Zhou, 2023. "An Integrated Strategy for Rescheduling High-Speed Train Operation under Single-Direction Disruption," Sustainability, MDPI, vol. 15(17), pages 1-31, August.
    6. Zhang, Yongxiang & D'Ariano, Andrea & He, Bisheng & Peng, Qiyuan, 2019. "Microscopic optimization model and algorithm for integrating train timetabling and track maintenance task scheduling," Transportation Research Part B: Methodological, Elsevier, vol. 127(C), pages 237-278.
    7. Wang, Yihui & Zhao, Kangqi & D’Ariano, Andrea & Niu, Ru & Li, Shukai & Luan, Xiaojie, 2021. "Real-time integrated train rescheduling and rolling stock circulation planning for a metro line under disruptions," Transportation Research Part B: Methodological, Elsevier, vol. 152(C), pages 87-117.
    8. Chen, Zebin & D’Ariano, Andrea & Li, Shukai & Tessitore, Marta Leonina & Yang, Lixing, 2024. "Robust dynamic train regulation integrated with stop-skipping strategy in urban rail networks: An outer approximation based solution method," Omega, Elsevier, vol. 128(C).
    9. Pellegrini, Paola & Marlière, Grégory & Rodriguez, Joaquin, 2014. "Optimal train routing and scheduling for managing traffic perturbations in complex junctions," Transportation Research Part B: Methodological, Elsevier, vol. 59(C), pages 58-80.
    10. Zhang, Huimin & Li, Shukai & Wang, Yihui & Yang, Lixing & Gao, Ziyou, 2021. "Collaborative real-time optimization strategy for train rescheduling and track emergency maintenance of high-speed railway: A Lagrangian relaxation-based decomposition algorithm," Omega, Elsevier, vol. 102(C).
    11. Lu, Gongyuan & Ning, Jia & Liu, Xiaobo & Nie, Yu (Marco), 2022. "Train platforming and rescheduling with flexible interlocking mechanisms: An aggregate approach," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 159(C).
    12. Jiateng Yin & Lixing Yang & Andrea D’Ariano & Tao Tang & Ziyou Gao, 2022. "Integrated Backup Rolling Stock Allocation and Timetable Rescheduling with Uncertain Time-Variant Passenger Demand Under Disruptive Events," INFORMS Journal on Computing, INFORMS, vol. 34(6), pages 3234-3258, November.
    13. Zhou, Leishan & Tong, Lu (Carol) & Chen, Junhua & Tang, Jinjin & Zhou, Xuesong, 2017. "Joint optimization of high-speed train timetables and speed profiles: A unified modeling approach using space-time-speed grid networks," Transportation Research Part B: Methodological, Elsevier, vol. 97(C), pages 157-181.
    14. Li, Wenqing & Ni, Shaoquan, 2022. "Train timetabling with the general learning environment and multi-agent deep reinforcement learning," Transportation Research Part B: Methodological, Elsevier, vol. 157(C), pages 230-251.
    15. Zhang, Qin & Lusby, Richard Martin & Shang, Pan & Zhu, Xiaoning, 2022. "A heuristic approach to integrate train timetabling, platforming, and railway network maintenance scheduling decisions," Transportation Research Part B: Methodological, Elsevier, vol. 158(C), pages 210-238.
    16. Xu, Xiaoming & Li, Keping & Yang, Lixing, 2015. "Scheduling heterogeneous train traffic on double tracks with efficient dispatching rules," Transportation Research Part B: Methodological, Elsevier, vol. 78(C), pages 364-384.
    17. Yang, Lin & Gao, Yuan & D’Ariano, Andrea & Xu, Suxiu, 2024. "Integrated optimization of train timetable and train unit circulation for a Y-type urban rail transit system with flexible train composition mode," Omega, Elsevier, vol. 122(C).
    18. Yang, Lixing & Qi, Jianguo & Li, Shukai & Gao, Yuan, 2016. "Collaborative optimization for train scheduling and train stop planning on high-speed railways," Omega, Elsevier, vol. 64(C), pages 57-76.
    19. Meng, Lingyun & Zhou, Xuesong, 2014. "Simultaneous train rerouting and rescheduling on an N-track network: A model reformulation with network-based cumulative flow variables," Transportation Research Part B: Methodological, Elsevier, vol. 67(C), pages 208-234.
    20. Lian, Deheng & Mo, Pengli & D’Ariano, Andrea & Gao, Ziyou & Yang, Lixing, 2024. "Energy-saving time allocation strategy with uncertain dwell times in urban rail transit: Two-stage stochastic model and nested dynamic programming framework," European Journal of Operational Research, Elsevier, vol. 317(1), pages 219-242.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:transe:v:194:y:2025:i:c:s1366554524004915. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/600244/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.