IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v390y2025ics0306261925006142.html
   My bibliography  Save this article

The impact of deep reinforcement learning-based traffic signal control on Emission reduction in urban Road networks empowered by cooperative vehicle-infrastructure systems

Author

Listed:
  • Shang, Wen-Long
  • Song, Xuewang
  • Xiang, Qiannian
  • Chen, Haibo
  • Elhajj, Mireille
  • Bi, Huibo
  • Wang, Kun
  • Ochieng, Washington

Abstract

The critical challenges of climate change have made carbon emission reduction an urgent global priority, with urban transportation systems (UTS) being significant contributors. Rapid urbanization and increased traffic demand have intensified congestion and carbon emissions. However, existing traffic signal control methods often rely on large historical data and pre-set signal timings, making it struggle to adapt dynamically to reduce emissions effectively. This study proposes a novel traffic signal control method based on Deep Reinforcement Learning (DRL), integrated with Cooperative Vehicle-Infrastructure Systems (CVIS) and a doubly Day-to-Day (DTD) Dynamic Traffic Assignment model, aimed at improving traffic efficiency and reducing carbon emissions. The DTD model simulates the urban road network as the training environment, while CVIS provides real-time traffic flow data that informs the DRL model's optimization of signal timings. Furthermore, the DTD model adjusts traveler departure times and route choices for significant emission reductions. The proposed DRL-based method significantly improves signal control efficiency and carbon emission reduction under complex traffic conditions. A case study on the Sioux Falls network indicates that the DRL-based traffic signal strategy outperforms traditional fixed-time control strategies, achieving CO2 emission reductions of 21 % to 27 % in various scenarios, particularly excelling in the SV scenario. Notably, emissions on high-traffic Link 28 (SV) were reduced by up to 54.9 %. This study underscores the potential of DRL in low-carbon traffic management and provides practical insights for the sustainable development of future traffic systems, offering robust solutions for emission reduction and efficient traffic management in UTS.

Suggested Citation

  • Shang, Wen-Long & Song, Xuewang & Xiang, Qiannian & Chen, Haibo & Elhajj, Mireille & Bi, Huibo & Wang, Kun & Ochieng, Washington, 2025. "The impact of deep reinforcement learning-based traffic signal control on Emission reduction in urban Road networks empowered by cooperative vehicle-infrastructure systems," Applied Energy, Elsevier, vol. 390(C).
  • Handle: RePEc:eee:appene:v:390:y:2025:i:c:s0306261925006142
    DOI: 10.1016/j.apenergy.2025.125884
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261925006142
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.apenergy.2025.125884?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Xiangwen Fu & Jing Cheng & Liqun Peng & Mi Zhou & Dan Tong & Denise L. Mauzerall, 2024. "Co-benefits of transport demand reductions from compact urban development in Chinese cities," Nature Sustainability, Nature, vol. 7(3), pages 294-304, March.
    2. Caroline Shaw & Simon Hales & Philippa Howden-Chapman & Richard Edwards, 2014. "Health co-benefits of climate change mitigation policies in the transport sector," Nature Climate Change, Nature, vol. 4(6), pages 427-433, June.
    3. Guo, Qiangqiang & Ban, Xuegang (Jeff), 2020. "Macroscopic fundamental diagram based perimeter control considering dynamic user equilibrium," Transportation Research Part B: Methodological, Elsevier, vol. 136(C), pages 87-109.
    4. Jia, Dongyao & Ngoduy, Dong, 2016. "Enhanced cooperative car-following traffic model with the combination of V2V and V2I communication," Transportation Research Part B: Methodological, Elsevier, vol. 90(C), pages 172-191.
    5. Viet Nguyen-Tien & Chengyu Zhang & Eric Strobl & Robert J. R. Elliott, 2025. "The closing longevity gap between battery electric vehicles and internal combustion vehicles in Great Britain," Nature Energy, Nature, vol. 10(3), pages 354-364, March.
    6. Fei, Xinyu & Wang, Xingmin & Yu, Xian & Feng, Yiheng & Liu, Henry & Shen, Siqian & Yin, Yafeng, 2023. "Traffic signal control under stochastic traffic demand and vehicle turning via decentralized decomposition approaches," European Journal of Operational Research, Elsevier, vol. 310(2), pages 712-736.
    7. Cheng, Yung-Hsiang & Chang, Yu-Hern & Lu, I.J., 2015. "Urban transportation energy and carbon dioxide emission reduction strategies," Applied Energy, Elsevier, vol. 157(C), pages 953-973.
    8. Shang, Wen-Long & Zhang, Mengxiao & Wu, Guoyuan & Yang, Lan & Fang, Shan & Ochieng, Washington, 2023. "Estimation of traffic energy consumption based on macro-micro modelling with sparse data from Connected and Automated Vehicles," Applied Energy, Elsevier, vol. 351(C).
    9. Woon, Kok Sin & Phuang, Zhen Xin & Taler, Jan & Varbanov, Petar Sabev & Chong, Cheng Tung & Klemeš, Jiří Jaromír & Lee, Chew Tin, 2023. "Recent advances in urban green energy development towards carbon emissions neutrality," Energy, Elsevier, vol. 267(C).
    10. Friesz, Terry L. & Han, Ke & Neto, Pedro A. & Meimand, Amir & Yao, Tao, 2013. "Dynamic user equilibrium based on a hydrodynamic model," Transportation Research Part B: Methodological, Elsevier, vol. 47(C), pages 102-126.
    11. Kucharski, Rafał & Gentile, Guido, 2019. "Simulation of rerouting phenomena in Dynamic Traffic Assignment with the Information Comply Model," Transportation Research Part B: Methodological, Elsevier, vol. 126(C), pages 414-441.
    12. Camilo Mora & Daniele Spirandelli & Erik C. Franklin & John Lynham & Michael B. Kantar & Wendy Miles & Charlotte Z. Smith & Kelle Freel & Jade Moy & Leo V. Louis & Evan W. Barba & Keith Bettinger & Ab, 2018. "Broad threat to humanity from cumulative climate hazards intensified by greenhouse gas emissions," Nature Climate Change, Nature, vol. 8(12), pages 1062-1071, December.
    13. Paul I. Richards, 1956. "Shock Waves on the Highway," Operations Research, INFORMS, vol. 4(1), pages 42-51, February.
    14. Sun, Bin & Zhang, Qijun & Wu, Zhong & Mao, Hongjun, 2024. "Impact of early-stage cooperative vehicle infrastructure systems on traffic and energy consumption under various traffic conditions: From application to policy," Energy, Elsevier, vol. 309(C).
    15. Shang, Wen-Long & Ling, Yantao & Ochieng, Washington & Yang, Linchuan & Gao, Xing & Ren, Qingzhong & Chen, Yilin & Cao, Mengqiu, 2024. "Driving forces of CO2 emissions from the transport, storage and postal sectors: A pathway to achieving carbon neutrality," Applied Energy, Elsevier, vol. 365(C).
    16. Shang, Wen-Long & Chen, Yishui & Yu, Qing & Song, Xuewang & Chen, Yanyan & Ma, Xiaolei & Chen, Xiqun & Tan, Zhijia & Huang, Jianling & Ochieng, Washington, 2023. "Spatio-temporal analysis of carbon footprints for urban public transport systems based on smart card data," Applied Energy, Elsevier, vol. 352(C).
    17. Chen, Xu & Wang, Xiaojun, 2016. "Effects of carbon emission reduction policies on transportation mode selections with stochastic demand," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 90(C), pages 196-205.
    18. Li, Chunjie & Xu, Chengcheng & Chen, Yusen & Li, Zhibin, 2024. "Development and experiment of an intelligent connected cooperative vehicle infrastructure system based on multiple V2I modes and BWM-IGR method," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 635(C).
    19. Bliemer, Michiel C.J. & Raadsen, Mark P.H., 2019. "Continuous-time general link transmission model with simplified fanning, Part I: Theory and link model formulation," Transportation Research Part B: Methodological, Elsevier, vol. 126(C), pages 442-470.
    20. Sangmin Park & Eum Han & Sungho Park & Harim Jeong & Ilsoo Yun, 2021. "Deep Q-network-based traffic signal control models," PLOS ONE, Public Library of Science, vol. 16(9), pages 1-14, September.
    21. Smith, Michael J & Viti, Francesco & Huang, Wei & Mounce, Richard, 2023. "With spatial queueing, the P0 responsive traffic signal control policy may fail to maximise network capacity even if queue storage capacities are very large," Transportation Research Part B: Methodological, Elsevier, vol. 177(C).
    22. Heleen L. Soest & Michel G. J. Elzen & Detlef P. Vuuren, 2021. "Net-zero emission targets for major emitting countries consistent with the Paris Agreement," Nature Communications, Nature, vol. 12(1), pages 1-9, December.
    23. Ke Han & Gabriel Eve & Terry L. Friesz, 2019. "Computing Dynamic User Equilibria on Large-Scale Networks with Software Implementation," Networks and Spatial Economics, Springer, vol. 19(3), pages 869-902, September.
    24. Shang, Wen-Long & Zhang, Junjie & Wang, Kun & Yang, Hangjun & Ochieng, Washington, 2024. "Can financial subsidy increase electric vehicle (EV) penetration---evidence from a quasi-natural experiment," Renewable and Sustainable Energy Reviews, Elsevier, vol. 190(PA).
    25. Yunpeng Sun & Qun Bao & Farhad Taghizadeh-Hesary, 2023. "Green finance, renewable energy development, and climate change: evidence from regions of China," Palgrave Communications, Palgrave Macmillan, vol. 10(1), pages 1-8, December.
    26. Raadsen, Mark P.H. & Bliemer, Michiel C.J., 2019. "Continuous-time general link transmission model with simplified fanning, Part II: Event-based algorithm for networks," Transportation Research Part B: Methodological, Elsevier, vol. 126(C), pages 471-501.
    27. Xue, Dabin & Du, Sen & Wang, Bing & Shang, Wen-Long & Avogadro, Nicolò & Ochieng, Washington Yotto, 2025. "Low-carbon benefits of aircraft adopting continuous descent operations," Applied Energy, Elsevier, vol. 383(C).
    28. Lisa Winkler & Drew Pearce & Jenny Nelson & Oytun Babacan, 2023. "The effect of sustainable mobility transition policies on cumulative urban transport emissions and energy demand," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    29. Fu, Haoran & Akamatsu, Takashi & Satsukawa, Koki & Wada, Kentaro, 2022. "Dynamic traffic assignment in a corridor network: Optimum versus equilibrium," Transportation Research Part B: Methodological, Elsevier, vol. 161(C), pages 218-246.
    30. Shang, Wen-Long & Chen, Jinyu & Bi, Huibo & Sui, Yi & Chen, Yanyan & Yu, Haitao, 2021. "Impacts of COVID-19 pandemic on user behaviors and environmental benefits of bike sharing: A big-data analysis," Applied Energy, Elsevier, vol. 285(C).
    31. Kumar, Amit & Peeta, Srinivas, 2015. "A day-to-day dynamical model for the evolution of path flows under disequilibrium of traffic networks with fixed demand," Transportation Research Part B: Methodological, Elsevier, vol. 80(C), pages 235-256.
    32. Jin, Wen-Long, 2015. "Continuous formulations and analytical properties of the link transmission model," Transportation Research Part B: Methodological, Elsevier, vol. 74(C), pages 88-103.
    33. Ukkusuri, Satish V. & Han, Lanshan & Doan, Kien, 2012. "Dynamic user equilibrium with a path based cell transmission model for general traffic networks," Transportation Research Part B: Methodological, Elsevier, vol. 46(10), pages 1657-1684.
    34. Deveci, Muhammet & Gokasar, Ilgin & Pamucar, Dragan & Zaidan, Aws Alaa & Wen, Xin & Gupta, Brij B., 2023. "Evaluation of Cooperative Intelligent Transportation System scenarios for resilience in transportation using type-2 neutrosophic fuzzy VIKOR," Transportation Research Part A: Policy and Practice, Elsevier, vol. 172(C).
    35. Osorio, Carolina & Nanduri, Kanchana, 2015. "Urban transportation emissions mitigation: Coupling high-resolution vehicular emissions and traffic models for traffic signal optimization," Transportation Research Part B: Methodological, Elsevier, vol. 81(P2), pages 520-538.
    36. Zhang, Pinchao & Qian, Sean, 2020. "Path-based system optimal dynamic traffic assignment: A subgradient approach," Transportation Research Part B: Methodological, Elsevier, vol. 134(C), pages 41-63.
    37. Zhang, Shaojun & Wu, Ye & Un, Puikei & Fu, Lixin & Hao, Jiming, 2016. "Modeling real-world fuel consumption and carbon dioxide emissions with high resolution for light-duty passenger vehicles in a traffic populated city," Energy, Elsevier, vol. 113(C), pages 461-471.
    38. Hoang, Nam H. & Vu, Hai L. & Panda, Manoj & Lo, Hong K., 2019. "A linear framework for dynamic user equilibrium traffic assignment in a single origin-destination capacitated network," Transportation Research Part B: Methodological, Elsevier, vol. 126(C), pages 329-352.
    39. Wen-Long Shang & Yanyan Chen & Xingang Li & Washington Y. Ochieng, 2020. "Resilience Analysis of Urban Road Networks Based on Adaptive Signal Controls: Day-to-Day Traffic Dynamics with Deep Reinforcement Learning," Complexity, Hindawi, vol. 2020, pages 1-19, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. František Kolovský & Ivana Kolingerová, 2022. "The Piecewise Constant/Linear Solution for Dynamic User Equilibrium," Networks and Spatial Economics, Springer, vol. 22(4), pages 737-765, December.
    2. Ke Han & Gabriel Eve & Terry L. Friesz, 2019. "Computing Dynamic User Equilibria on Large-Scale Networks with Software Implementation," Networks and Spatial Economics, Springer, vol. 19(3), pages 869-902, September.
    3. Du, Jinxiao & Ma, Wei, 2024. "Maximin headway control of automated vehicles for system optimal dynamic traffic assignment in general networks," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 188(C).
    4. Yu, Hao & Ma, Rui & Zhang, H. Michael, 2018. "Optimal traffic signal control under dynamic user equilibrium and link constraints in a general network," Transportation Research Part B: Methodological, Elsevier, vol. 110(C), pages 302-325.
    5. Wang, Guanfeng & Jia, Hongfei & Feng, Tao & Tian, Jingjing & Wu, Ruiyi & Gao, Heyao & Liu, Chao, 2024. "Modelling the dual dynamic traffic flow evolution with information perception differences between human-driven vehicles and connected autonomous vehicles," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 640(C).
    6. Bliemer, Michiel C.J. & Raadsen, Mark P.H., 2020. "Static traffic assignment with residual queues and spillback," Transportation Research Part B: Methodological, Elsevier, vol. 132(C), pages 303-319.
    7. Ngoduy, D. & Hoang, N.H. & Vu, H.L. & Watling, D., 2016. "Optimal queue placement in dynamic system optimum solutions for single origin-destination traffic networks," Transportation Research Part B: Methodological, Elsevier, vol. 92(PB), pages 148-169.
    8. Bliemer, Michiel C.J. & Raadsen, Mark P.H., 2019. "Continuous-time general link transmission model with simplified fanning, Part I: Theory and link model formulation," Transportation Research Part B: Methodological, Elsevier, vol. 126(C), pages 442-470.
    9. Li, Xinghua & Zhang, Xinyuan & Qian, Xinwu & Zhao, Cong & Guo, Yuntao & Peeta, Srinivas, 2024. "Beyond centralization: Non-cooperative perimeter control with extended mean-field reinforcement learning in urban road networks," Transportation Research Part B: Methodological, Elsevier, vol. 186(C).
    10. Han, Ke & Friesz, Terry L. & Szeto, W.Y. & Liu, Hongcheng, 2015. "Elastic demand dynamic network user equilibrium: Formulation, existence and computation," Transportation Research Part B: Methodological, Elsevier, vol. 81(P1), pages 183-209.
    11. Samitha Samaranayake & Walid Krichene & Jack Reilly & Maria Laura Delle Monache & Paola Goatin & Alexandre Bayen, 2018. "Discrete-Time System Optimal Dynamic Traffic Assignment (SO-DTA) with Partial Control for Physical Queuing Networks," Transportation Science, INFORMS, vol. 52(4), pages 982-1001, August.
    12. Wang, Yi & Szeto, W.Y. & Han, Ke & Friesz, Terry L., 2018. "Dynamic traffic assignment: A review of the methodological advances for environmentally sustainable road transportation applications," Transportation Research Part B: Methodological, Elsevier, vol. 111(C), pages 370-394.
    13. Long, Jiancheng & Szeto, W.Y. & Huang, Hai-Jun & Gao, Ziyou, 2015. "An intersection-movement-based stochastic dynamic user optimal route choice model for assessing network performance," Transportation Research Part B: Methodological, Elsevier, vol. 74(C), pages 182-217.
    14. Babak Javani & Abbas Babazadeh, 2020. "Path-Based Dynamic User Equilibrium Model with Applications to Strategic Transportation Planning," Networks and Spatial Economics, Springer, vol. 20(2), pages 329-366, June.
    15. Xue, Dabin & Du, Sen & Wang, Bing & Shang, Wen-Long & Avogadro, Nicolò & Ochieng, Washington Yotto, 2025. "Low-carbon benefits of aircraft adopting continuous descent operations," Applied Energy, Elsevier, vol. 383(C).
    16. Tilg, Gabriel & Ambühl, Lukas & Batista, Sergio & Menendez, Monica & Busch, Fritz, 2021. "On the application of variational theory to urban networks," Transportation Research Part B: Methodological, Elsevier, vol. 150(C), pages 435-456.
    17. Himpe, Willem & Corthout, Ruben & Tampère, M.J. Chris, 2016. "An efficient iterative link transmission model," Transportation Research Part B: Methodological, Elsevier, vol. 92(PB), pages 170-190.
    18. Chou, Chang-Chi & Chiang, Wen-Chu & Chen, Albert Y., 2022. "Emergency medical response in mass casualty incidents considering the traffic congestions in proximity on-site and hospital delays," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 158(C).
    19. Wan, Tianyi & Fu, Hao & Li, Xiaoshan & Wu, Fan & Luo, Cong & Zhang, Liqi, 2025. "Assessment of decarbonization pathway for Chinese road transport sector based on transportation-energy integration systems framework," Energy, Elsevier, vol. 317(C).
    20. Jin, Wen-Long, 2018. "Unifiable multi-commodity kinematic wave model," Transportation Research Part B: Methodological, Elsevier, vol. 117(PB), pages 639-659.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:390:y:2025:i:c:s0306261925006142. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.