IDEAS home Printed from https://ideas.repec.org/a/eee/transe/v166y2022ics1366554522002514.html
   My bibliography  Save this article

Deep reinforcement learning for dynamic incident-responsive traffic information dissemination

Author

Listed:
  • Xie, Jiaohong
  • Yang, Zhenyu
  • Lai, Xiongfei
  • Liu, Yang
  • Yang, Xiao Bo
  • Teng, Teck-Hou
  • Tham, Chen-Khong

Abstract

This study is concerned with the optimal dynamical information dissemination (DID) problem in a transportation network interrupted by traffic incidents. Optimizing system performance with DID after road incidents is challenging because of the uncertainty in traffic flow variation and travelers’ heterogeneous responses to information. To address the problem, we consider a traffic manager who aims to improve the system performance by dynamically generating and disseminating information to road users in a time period after an incident happens. We develop a decision tool for obtaining DID strategy based on double deep Q-learning (DDQL) for the traffic manager, aiming at finding an optimal DID strategy. The decision tool is integrated with traffic sensors which collect traffic data in real time. With advanced traveler information systems, the DID system dynamically sends out various types of information to users according to the current and anticipated traffic states so as to minimize congestion and enhance road network capacity. In particular, the proposed DDQL method utilizes a double deep Q-network (DQN) structure to learn the state–action values. To test and evaluate the performance of the decision tool, we develop a microscopic simulation model of a real road network in the Serangoon area of Singapore in PTV VISSIM and calibrate the model with real historical traffic data. We train and compare the DDQL controller model with different reward signals, including the weighted sum of the average speed and queue delay, total traffic flow, and average travel time. Numerical experiments demonstrate the good performance of the proposed DDQL-based DID approach in improving the congestion and other performance metrics of the expressway. The robustness and generalizability of the DDQL agent are also verified by evaluating the algorithm performance in environments with different traffic demand patterns and driving behavior profiles.

Suggested Citation

  • Xie, Jiaohong & Yang, Zhenyu & Lai, Xiongfei & Liu, Yang & Yang, Xiao Bo & Teng, Teck-Hou & Tham, Chen-Khong, 2022. "Deep reinforcement learning for dynamic incident-responsive traffic information dissemination," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 166(C).
  • Handle: RePEc:eee:transe:v:166:y:2022:i:c:s1366554522002514
    DOI: 10.1016/j.tre.2022.102871
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1366554522002514
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.tre.2022.102871?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Basso, Rafael & Kulcsár, Balázs & Sanchez-Diaz, Ivan & Qu, Xiaobo, 2022. "Dynamic stochastic electric vehicle routing with safe reinforcement learning," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 157(C).
    2. Pi, Xidong & Qian, Zhen (Sean), 2017. "A stochastic optimal control approach for real-time traffic routing considering demand uncertainties and travelers’ choice heterogeneity," Transportation Research Part B: Methodological, Elsevier, vol. 104(C), pages 710-732.
    3. Khoo, Hooi Ling & Asitha, K.S., 2016. "An impact analysis of traffic image information system on driver travel choice," Transportation Research Part A: Policy and Practice, Elsevier, vol. 88(C), pages 175-194.
    4. Constantinos Antoniou & Haris N. Koutsopoulos & Moshe Ben-Akiva & Akhilendra S. Chauhan, 2011. "Evaluation of diversion strategies using dynamic traffic assignment," Transportation Planning and Technology, Taylor & Francis Journals, vol. 34(3), pages 199-216, February.
    5. Chen, Danjue & Ahn, Soyoung & Hegyi, Andreas, 2014. "Variable speed limit control for steady and oscillatory queues at fixed freeway bottlenecks," Transportation Research Part B: Methodological, Elsevier, vol. 70(C), pages 340-358.
    6. Wang, Yineng & Li, Meng & Lin, Xi & He, Fang, 2021. "Online operations strategies for automated multistory parking facilities," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 145(C).
    7. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    8. Asadi, Amin & Nurre Pinkley, Sarah, 2021. "A stochastic scheduling, allocation, and inventory replenishment problem for battery swap stations," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 146(C).
    9. Minghui Ma & Qingfang Yang & Shidong Liang & Yashi Wang, 2016. "A New Coordinated Control Method on the Intersection of Traffic Region," Discrete Dynamics in Nature and Society, Hindawi, vol. 2016, pages 1-10, May.
    10. Zhao, Wenjing & Ma, Zhuanglin & Yang, Kui & Huang, Helai & Monsuur, Fredrik & Lee, Jaeyoung, 2020. "Impacts of variable message signs on en-route route choice behavior," Transportation Research Part A: Policy and Practice, Elsevier, vol. 139(C), pages 335-349.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fescioglu-Unver, Nilgun & Yıldız Aktaş, Melike, 2023. "Electric vehicle charging service operations: A review of machine learning applications for infrastructure planning, control, pricing and routing," Renewable and Sustainable Energy Reviews, Elsevier, vol. 188(C).
    2. Yan, Yimo & Chow, Andy H.F. & Ho, Chin Pang & Kuo, Yong-Hong & Wu, Qihao & Ying, Chengshuo, 2022. "Reinforcement learning for logistics and supply chain management: Methodologies, state of the art, and future opportunities," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 162(C).
    3. Liu, Zeyu & Li, Xueping & Khojandi, Anahita, 2022. "The flying sidekick traveling salesman problem with stochastic travel time: A reinforcement learning approach," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 164(C).
    4. Tulika Saha & Sriparna Saha & Pushpak Bhattacharyya, 2020. "Towards sentiment aided dialogue policy learning for multi-intent conversations using hierarchical reinforcement learning," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-28, July.
    5. Simona Mikšíková & David Ulčák & František Kuda, 2022. "Analysis of Malfunctions in Selected Parking Systems in the Czech Republic," Sustainability, MDPI, vol. 14(3), pages 1-10, February.
    6. Mahmoud Mahfouz & Angelos Filos & Cyrine Chtourou & Joshua Lockhart & Samuel Assefa & Manuela Veloso & Danilo Mandic & Tucker Balch, 2019. "On the Importance of Opponent Modeling in Auction Markets," Papers 1911.12816, arXiv.org.
    7. Imen Azzouz & Wiem Fekih Hassen, 2023. "Optimization of Electric Vehicles Charging Scheduling Based on Deep Reinforcement Learning: A Decentralized Approach," Energies, MDPI, vol. 16(24), pages 1-18, December.
    8. Jacob W. Crandall & Mayada Oudah & Tennom & Fatimah Ishowo-Oloko & Sherief Abdallah & Jean-François Bonnefon & Manuel Cebrian & Azim Shariff & Michael A. Goodrich & Iyad Rahwan, 2018. "Cooperating with machines," Nature Communications, Nature, vol. 9(1), pages 1-12, December.
      • Abdallah, Sherief & Bonnefon, Jean-François & Cebrian, Manuel & Crandall, Jacob W. & Ishowo-Oloko, Fatimah & Oudah, Mayada & Rahwan, Iyad & Shariff, Azim & Tennom,, 2017. "Cooperating with Machines," TSE Working Papers 17-806, Toulouse School of Economics (TSE).
      • Abdallah, Sherief & Bonnefon, Jean-François & Cebrian, Manuel & Crandall, Jacob W. & Ishowo-Oloko, Fatimah & Oudah, Mayada & Rahwan, Iyad & Shariff, Azim & Tennom,, 2017. "Cooperating with Machines," IAST Working Papers 17-68, Institute for Advanced Study in Toulouse (IAST).
      • Jacob Crandall & Mayada Oudah & Fatimah Ishowo-Oloko Tennom & Fatimah Ishowo-Oloko & Sherief Abdallah & Jean-François Bonnefon & Manuel Cebrian & Azim Shariff & Michael Goodrich & Iyad Rahwan, 2018. "Cooperating with machines," Post-Print hal-01897802, HAL.
    9. Sun, Alexander Y., 2020. "Optimal carbon storage reservoir management through deep reinforcement learning," Applied Energy, Elsevier, vol. 278(C).
    10. Yassine Chemingui & Adel Gastli & Omar Ellabban, 2020. "Reinforcement Learning-Based School Energy Management System," Energies, MDPI, vol. 13(23), pages 1-21, December.
    11. Woo Jae Byun & Bumkyu Choi & Seongmin Kim & Joohyun Jo, 2023. "Practical Application of Deep Reinforcement Learning to Optimal Trade Execution," FinTech, MDPI, vol. 2(3), pages 1-16, June.
    12. Lu, Yu & Xiang, Yue & Huang, Yuan & Yu, Bin & Weng, Liguo & Liu, Junyong, 2023. "Deep reinforcement learning based optimal scheduling of active distribution system considering distributed generation, energy storage and flexible load," Energy, Elsevier, vol. 271(C).
    13. Yuhong Wang & Lei Chen & Hong Zhou & Xu Zhou & Zongsheng Zheng & Qi Zeng & Li Jiang & Liang Lu, 2021. "Flexible Transmission Network Expansion Planning Based on DQN Algorithm," Energies, MDPI, vol. 14(7), pages 1-21, April.
    14. Huang, Ruchen & He, Hongwen & Gao, Miaojue, 2023. "Training-efficient and cost-optimal energy management for fuel cell hybrid electric bus based on a novel distributed deep reinforcement learning framework," Applied Energy, Elsevier, vol. 346(C).
    15. Michelle M. LaMar, 2018. "Markov Decision Process Measurement Model," Psychometrika, Springer;The Psychometric Society, vol. 83(1), pages 67-88, March.
    16. Zichen Lu & Ying Yan, 2024. "Temperature Control of Fuel Cell Based on PEI-DDPG," Energies, MDPI, vol. 17(7), pages 1-19, April.
    17. Yang, Ting & Zhao, Liyuan & Li, Wei & Zomaya, Albert Y., 2021. "Dynamic energy dispatch strategy for integrated energy system based on improved deep reinforcement learning," Energy, Elsevier, vol. 235(C).
    18. Wang, Xuan & Shu, Gequn & Tian, Hua & Wang, Rui & Cai, Jinwen, 2020. "Operation performance comparison of CCHP systems with cascade waste heat recovery systems by simulation and operation optimisation," Energy, Elsevier, vol. 206(C).
    19. Wang, Yi & Qiu, Dawei & Sun, Mingyang & Strbac, Goran & Gao, Zhiwei, 2023. "Secure energy management of multi-energy microgrid: A physical-informed safe reinforcement learning approach," Applied Energy, Elsevier, vol. 335(C).
    20. Parvez Farazi, Nahid & Zou, Bo & Tulabandhula, Theja, 2022. "Dynamic On-Demand Crowdshipping Using Constrained and Heuristics-Embedded Double Dueling Deep Q-Network," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 166(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:transe:v:166:y:2022:i:c:s1366554522002514. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/600244/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.