IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v330y2025ics0360544225023722.html

Towards sustainable and intelligent urban transportation: A novel deep transfer reinforcement learning framework for eco-driving of fuel cell buses

Author

Listed:
  • Huang, Ruchen
  • He, Hongwen
  • Su, Qicong
  • Wu, Jingda

Abstract

Eco-driving is a sustainable technology that optimizes both energy management and speed planning for electrified vehicles. Particularly when combined with emerging deep reinforcement learning (DRL) techniques, eco-driving strategies (EDSs) can be more intelligent. However, current research on eco-driving, namely the holistic solution, lags behind the advancements in its sub-problem namely energy management, and the development of DRL-based EDSs remains time-consuming. Since energy management is a sub-task of eco-driving, it offers a potential way to rapidly develop EDSs by reusing pre-trained energy management strategies (EMSs). Based on this, this paper proposes an expedited method for developing soft actor-critic (SAC) based EDSs for fuel cell buses (FCBs) in the vehicle-following scenario. To ensure that SAC-based EMSs can be effectively transferred to EDSs, an innovative heterogeneous deep transfer reinforcement learning framework is designed. Within this framework, all the knowledge learned in the source EMS can be transferred and reused by the target EDS. More importantly, the transferability of heterogeneous deep neural networks and heterogeneous experience replay buffers is particularly verified. Simulation results show that the proposed framework provides a 71.01 % acceleration in convergence speed and a 7.30 % improvement in fuel economy. This article contributes to correlating different optimization tasks of FCBs through advanced artificial intelligence technologies.

Suggested Citation

  • Huang, Ruchen & He, Hongwen & Su, Qicong & Wu, Jingda, 2025. "Towards sustainable and intelligent urban transportation: A novel deep transfer reinforcement learning framework for eco-driving of fuel cell buses," Energy, Elsevier, vol. 330(C).
    • Handle: RePEc:eee:energy:v:330:y:2025:i:c:s0360544225023722
    DOI: 10.1016/j.energy.2025.136730
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544225023722
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2025.136730?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Liu, Weirong & Yao, Pengfei & Wu, Yue & Duan, Lijun & Li, Heng & Peng, Jun, 2025. "Imitation reinforcement learning energy management for electric vehicles with hybrid energy storage system," Applied Energy, Elsevier, vol. 378(PA).
    2. Huang, Ruchen & He, Hongwen & Su, Qicong, 2024. "Towards a fossil-free urban transport system: An intelligent cross-type transferable energy management framework based on deep transfer reinforcement learning," Applied Energy, Elsevier, vol. 363(C).
    3. Christina V. Theodoris & Ling Xiao & Anant Chopra & Mark D. Chaffin & Zeina R. Al Sayed & Matthew C. Hill & Helene Mantineo & Elizabeth M. Brydon & Zexian Zeng & X. Shirley Liu & Patrick T. Ellinor, 2023. "Transfer learning enables predictions in network biology," Nature, Nature, vol. 618(7965), pages 616-624, June.
    4. Guan, Kaifu & Huang, Zhiwu & Gao, Yang & Wu, Yue & Li, Fei & Li, Heng, 2025. "Towards adaptive deep reinforcement learning energy management for electric vehicles: An online updating approach," Energy, Elsevier, vol. 325(C).
    5. Liu, Bo & Sun, Chao & Wang, Bo & Liang, Weiqiang & Ren, Qiang & Li, Junqiu & Sun, Fengchun, 2022. "Bi-level convex optimization of eco-driving for connected Fuel Cell Hybrid Electric Vehicles through signalized intersections," Energy, Elsevier, vol. 252(C).
    6. Huang, Ruchen & He, Hongwen & Su, Qicong, 2024. "An intelligent full-knowledge transferable collaborative eco-driving framework based on improved soft actor-critic algorithm," Applied Energy, Elsevier, vol. 375(C).
    7. Wang, Kang & Wang, Haixin & Yang, Zihao & Feng, Jiawei & Li, Yanzhen & Yang, Junyou & Chen, Zhe, 2023. "A transfer learning method for electric vehicles charging strategy based on deep reinforcement learning," Applied Energy, Elsevier, vol. 343(C).
    8. Lee, Heeyun & Kim, Kyunghyun & Kim, Namwook & Cha, Suk Won, 2022. "Energy efficient speed planning of electric vehicles for car-following scenario using model-based reinforcement learning," Applied Energy, Elsevier, vol. 313(C).
    9. Peter R. Wurman & Samuel Barrett & Kenta Kawamoto & James MacGlashan & Kaushik Subramanian & Thomas J. Walsh & Roberto Capobianco & Alisa Devlic & Franziska Eckert & Florian Fuchs & Leilani Gilpin & P, 2022. "Outracing champion Gran Turismo drivers with deep reinforcement learning," Nature, Nature, vol. 602(7896), pages 223-228, February.
    10. Zhang, Hailong & Peng, Jiankun & Dong, Hanxuan & Tan, Huachun & Ding, Fan, 2023. "Hierarchical reinforcement learning based energy management strategy of plug-in hybrid electric vehicle for ecological car-following process," Applied Energy, Elsevier, vol. 333(C).
    11. Huang, Ruchen & He, Hongwen & Su, Qicong & Härtl, Martin & Jaensch, Malte, 2025. "Type- and task-crossing energy management for fuel cell vehicles with longevity consideration: A heterogeneous deep transfer reinforcement learning framework," Applied Energy, Elsevier, vol. 377(PC).
    12. Zhang, Chuntao & Huang, Wenhui & Zhou, Xingyu & Lv, Chen & Sun, Chao, 2024. "Expert-demonstration-augmented reinforcement learning for lane-change-aware eco-driving traversing consecutive traffic lights," Energy, Elsevier, vol. 286(C).
    13. Tan, Yingqi & Xu, Jingyi & Ma, Junyi & Li, Zirui & Chen, Huiyan & Xi, Junqiang & Liu, Haiou, 2024. "A transferable perception-guided EMS for series hybrid electric unmanned tracked vehicles," Energy, Elsevier, vol. 306(C).
    14. Maino, Claudio & Misul, Daniela & Musa, Alessia & Spessa, Ezio, 2021. "Optimal mesh discretization of the dynamic programming for hybrid electric vehicles," Applied Energy, Elsevier, vol. 292(C).
    15. Huang, Ruchen & He, Hongwen & Su, Qicong & Härtl, Martin & Jaensch, Malte, 2024. "Enabling cross-type full-knowledge transferable energy management for hybrid electric vehicles via deep transfer reinforcement learning," Energy, Elsevier, vol. 305(C).
    16. Li, Fangyi & Zhao, Zhewen & Li, Fei & Xie, Wu, 2024. "Carbon reduction effects of electric delivery vehicle adoption in Chinese cities: Gradient pattern and scenario analysis," Energy, Elsevier, vol. 309(C).
    17. Huang, Ruchen & He, Hongwen & Zhao, Xuyang & Wang, Yunlong & Li, Menglin, 2022. "Battery health-aware and naturalistic data-driven energy management for hybrid electric bus based on TD3 deep reinforcement learning algorithm," Applied Energy, Elsevier, vol. 321(C).
    18. Wu, Jingda & Huang, Chao & He, Hongwen & Huang, Hailong, 2024. "Confidence-aware reinforcement learning for energy management of electrified vehicles," Renewable and Sustainable Energy Reviews, Elsevier, vol. 191(C).
    19. Tang, Tianfeng & Peng, Qianlong & Shi, Qing & Peng, Qingguo & Zhao, Jin & Chen, Chaoyi & Wang, Guangwei, 2024. "Energy management of fuel cell hybrid electric bus in mountainous regions: A deep reinforcement learning approach considering terrain characteristics," Energy, Elsevier, vol. 311(C).
    20. Sun, Chao & Zhang, Chuntao & Sun, Fengchun & Zhou, Xingyu, 2022. "Stochastic co-optimization of speed planning and powertrain control with dynamic probabilistic constraints for safe and ecological driving," Applied Energy, Elsevier, vol. 325(C).
    21. Liang, Zhaowen & Ruan, Jiageng & Wang, Zhenpo & Liu, Kai & Li, Bin, 2024. "Soft actor-critic-based EMS design for dual motor battery electric bus," Energy, Elsevier, vol. 288(C).
    22. Chang, Chengcheng & Zhao, Wanzhong & Wang, Chunyan & Luan, Zhongkai, 2023. "An energy management strategy of deep reinforcement learning based on multi-agent architecture under self-generating conditions," Energy, Elsevier, vol. 283(C).
    23. Liu, Yonggang & Huang, Bin & Yang, Yang & Lei, Zhenzhen & Zhang, Yuanjian & Chen, Zheng, 2022. "Hierarchical speed planning and energy management for autonomous plug-in hybrid electric vehicle in vehicle-following environment," Energy, Elsevier, vol. 260(C).
    24. Zhou, Jianhao & Xue, Siwu & Xue, Yuan & Liao, Yuhui & Liu, Jun & Zhao, Wanzhong, 2021. "A novel energy management strategy of hybrid electric vehicle via an improved TD3 deep reinforcement learning," Energy, Elsevier, vol. 224(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Huang, Ruchen & He, Hongwen, 2025. "UpdatingEMS: An online updating framework for deep reinforcement learning-based energy management of fuel cell hybrid electric bus with integrated transfer learning," Applied Energy, Elsevier, vol. 402(PA).
    2. Huang, Ruchen & He, Hongwen & Su, Qicong, 2024. "Smart energy management for hybrid electric bus via improved soft actor-critic algorithm in a heuristic learning framework," Energy, Elsevier, vol. 309(C).
    3. Li, Xueliang & Liu, Yilong & Yan, Mei & Tian, Dayu & Yang, Shujun & Peng, Zengxiong, 2026. "Data-driven a convergence-enhanced fusion energy management strategy based on teacher agent guidance for hybrid electric vehicles," Applied Energy, Elsevier, vol. 404(C).
    4. Fang, Shuo & Hu, Shuangxi & Liu, Yuntao & Zhao, Chunhui & Wang, Ying, 2025. "Power management unit with maximum-efficiency-point-tracking to enhance the efficiency of micro DMFC stack," Energy, Elsevier, vol. 315(C).
    5. Huang, Ruchen & He, Hongwen & Su, Qicong & Härtl, Martin & Jaensch, Malte, 2025. "Type- and task-crossing energy management for fuel cell vehicles with longevity consideration: A heterogeneous deep transfer reinforcement learning framework," Applied Energy, Elsevier, vol. 377(PC).
    6. Nie, Zhigen & Feng, Yaxing & Lian, Yufeng, 2025. "Deep reinforcement learning-based hierarchical control strategy for energy management of intelligent fuel cell hybrid electric vehicles," Energy, Elsevier, vol. 326(C).
    7. Mahmud, Sakib & Sayed, Aya Nabil & Himeur, Yassine & Nhlabatsi, Armstrong & Bensaali, Faycal, 2026. "A comprehensive review of deep reinforcement learning applications from centralized power generation to modern energy internet frameworks," Renewable and Sustainable Energy Reviews, Elsevier, vol. 226(PE).
    8. Huang, Ruchen & He, Hongwen & Su, Qicong, 2024. "An intelligent full-knowledge transferable collaborative eco-driving framework based on improved soft actor-critic algorithm," Applied Energy, Elsevier, vol. 375(C).
    9. Li, Menglin & Yin, Long & Yan, Mei & Wu, Jingda & He, Hongwe & Jia, Chunchun, 2024. "Hierarchical intelligent energy-saving control strategy for fuel cell hybrid electric buses based on traffic flow predictions," Energy, Elsevier, vol. 304(C).
    10. Zhou, Xingyu & Guo, Yuekai & Hao, Wenmei & Zhang, Xingyong & Wang, Wenwei, 2025. "Kinematic state adaptive based novel powertrain for the next generation of vehicle electrification: Design optimization and simplification," Applied Energy, Elsevier, vol. 395(C).
    11. Zhao, Ganglei & Li, Xiang & Liu, Changxing & Zhang, Huiliu & Li, Po, 2026. "Experimental validation of real-time energy management for hybrid energy storage systems based on predictive wavelet transforms," Applied Energy, Elsevier, vol. 405(C).
    12. Yu, Qiang & He, Xionglin & Chen, Yongji & Jiang, Zihong & Tan, Yilin & Liu, Longze & Xie, Bin & Wen, Changkai, 2025. "Multi-objective optimization for energy-efficient management of electric Tractors via hybrid energy storage systems and scenario recognition," Applied Energy, Elsevier, vol. 391(C).
    13. Nie, Zhigen & Song, Hao & Lian, Yufeng & Shi, Zhuangfeng, 2025. "Hierarchical optimization of SAC-driven speed planning and energy management in intelligent fuel cell hybrid vehicle platoons under complex traffic environments," Energy, Elsevier, vol. 334(C).
    14. He, Hongwen & Su, Qicong & Huang, Ruchen & Niu, Zegong, 2024. "Enabling intelligent transferable energy management of series hybrid electric tracked vehicle across motion dimensions via soft actor-critic algorithm," Energy, Elsevier, vol. 294(C).
    15. Zhang, Wei & Li, Jinchen & Ma, Hongfeng & Niu, Chengyong & Wang, Jixin & Yang, Zhiyu & Li, Lijun & Huang, Linsen, 2025. "Intelligent energy management in hybrid electric cycling work vehicles: A multi-task learning approach," Energy, Elsevier, vol. 326(C).
    16. Huang, Ruchen & He, Hongwen & Su, Qicong & Härtl, Martin & Jaensch, Malte, 2024. "Enabling cross-type full-knowledge transferable energy management for hybrid electric vehicles via deep transfer reinforcement learning," Energy, Elsevier, vol. 305(C).
    17. Hu, Dong & Huang, Chao & Wu, Jingda & Wei, Henglai & Pi, Dawei, 2025. "Enhancing data-driven energy management strategy via digital expert guidance for electrified vehicles," Applied Energy, Elsevier, vol. 381(C).
    18. Liu, Hui & You, Congwen & Han, Lijin & Yang, Ningkang & Liu, Baoshuai, 2025. "Off-road hybrid electric vehicle energy management strategy using multi-agent soft actor-critic with collaborative-independent algorithm," Energy, Elsevier, vol. 328(C).
    19. Dai, Churong & Zuo, Wei & Li, Qingqing & Zhou, Kun & Huang, Yuhan & Zhang, Guangde & E, Jiaqiang, 2024. "Energy conversion efficiency improvement studies on the hydrogen-fueled micro planar combustor with multi-baffles for thermophotovoltaic applications," Energy, Elsevier, vol. 313(C).
    20. Lei, Nuo & Zhang, Hao & Hu, Jingjing & Hu, Zunyan & Wang, Zhi, 2025. "Sim-to-real design and development of reinforcement learning-based energy management strategies for fuel cell electric vehicles," Applied Energy, Elsevier, vol. 393(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:330:y:2025:i:c:s0360544225023722. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.