IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v285y2023ics0360544223028438.html
   My bibliography  Save this article

Research on energy-saving driving control of hydrogen fuel bus based on deep reinforcement learning in freeway ramp weaving area

Author

Listed:
  • Peng, Jiankun
  • Shen, Yang
  • Wu, ChangCheng
  • Wang, Chunhai
  • Yi, Fengyan
  • Ma, Chunye

Abstract

In the intelligent network traffic scenario, the convenient acquisition of microscopic vehicle states and global traffic states can help solve the problems of vehicle driving and energy management in complex traffic environments. This paper proposes a new energy management method for the hydrogen fuel cell bus based on the double-layer deep deterministic policy gradient (DDPG). Combined with the SUMO simulation platform, a double-layer deep reinforcement learning (D-DRL) architecture based on DDPG is designed to improve control accuracy and training speed. In the upper D-DRL, the Agent handles the effects of complex traffic environments to control the reasonable speed of the vehicle and keep it running smoothly to reduce the loss of energy caused by speed changes, compared with the SUMO-IDM model, the maximum-minimum velocity difference was reduced by 21 % and the acceleration and acceleration change was reduced by 7.9 % and 19 %, After the Agent receives the output speed from the upper layer, it distributes the power between fuel cell and power cell. Compared with the DP algorithm, it keeps the SOC at a higher level, the hydrogen consumption level reaches 93.25 %, and the fluctuation amplitude decreases by 42.09 %, effectively improving fuel cell durability.

Suggested Citation

  • Peng, Jiankun & Shen, Yang & Wu, ChangCheng & Wang, Chunhai & Yi, Fengyan & Ma, Chunye, 2023. "Research on energy-saving driving control of hydrogen fuel bus based on deep reinforcement learning in freeway ramp weaving area," Energy, Elsevier, vol. 285(C).
    • Handle: RePEc:eee:energy:v:285:y:2023:i:c:s0360544223028438
    DOI: 10.1016/j.energy.2023.129449
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544223028438
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2023.129449?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Xie, Shaobo & Hu, Xiaosong & Xin, Zongke & Brighton, James, 2019. "Pontryagin’s Minimum Principle based model predictive control of energy management for a plug-in hybrid electric bus," Applied Energy, Elsevier, vol. 236(C), pages 893-905.
    2. Lian, Renzong & Peng, Jiankun & Wu, Yuankai & Tan, Huachun & Zhang, Hailong, 2020. "Rule-interposing deep reinforcement learning based energy management strategy for power-split hybrid electric vehicle," Energy, Elsevier, vol. 197(C).
    3. Hou, Cong & Ouyang, Minggao & Xu, Liangfei & Wang, Hewu, 2014. "Approximate Pontryagin’s minimum principle applied to the energy management of plug-in hybrid electric vehicles," Applied Energy, Elsevier, vol. 115(C), pages 174-189.
    4. Huang, Ruchen & He, Hongwen & Gao, Miaojue, 2023. "Training-efficient and cost-optimal energy management for fuel cell hybrid electric bus based on a novel distributed deep reinforcement learning framework," Applied Energy, Elsevier, vol. 346(C).
    5. Ioan-Sorin Sorlei & Nicu Bizon & Phatiphat Thounthong & Mihai Varlam & Elena Carcadea & Mihai Culcer & Mariana Iliescu & Mircea Raceanu, 2021. "Fuel Cell Electric Vehicles—A Brief Review of Current Topologies and Energy Management Strategies," Energies, MDPI, vol. 14(1), pages 1-29, January.
    6. He, Hongwen & Wang, Yunlong & Han, Ruoyan & Han, Mo & Bai, Yunfei & Liu, Qingwu, 2021. "An improved MPC-based energy management strategy for hybrid vehicles using V2V and V2I communications," Energy, Elsevier, vol. 225(C).
    7. Barkenbus, Jack N., 2010. "Eco-driving: An overlooked climate change initiative," Energy Policy, Elsevier, vol. 38(2), pages 762-769, February.
    8. Song, Ke & Wang, Xiaodi & Li, Feiqiang & Sorrentino, Marco & Zheng, Bailin, 2020. "Pontryagin’s minimum principle-based real-time energy management strategy for fuel cell hybrid electric vehicle considering both fuel economy and power source durability," Energy, Elsevier, vol. 205(C).
    9. Baroutaji, Ahmad & Wilberforce, Tabbi & Ramadan, Mohamad & Olabi, Abdul Ghani, 2019. "Comprehensive investigation on hydrogen and fuel cell technology in the aviation and aerospace sectors," Renewable and Sustainable Energy Reviews, Elsevier, vol. 106(C), pages 31-40.
    10. Wang, Hanchen & Ye, Yiming & Zhang, Jiangfeng & Xu, Bin, 2023. "A comparative study of 13 deep reinforcement learning based energy management methods for a hybrid electric vehicle," Energy, Elsevier, vol. 266(C).
    11. Nègre, Julien & Delhomme, Patricia, 2017. "Drivers’ self-perceptions about being an eco-driver according to their concern for the environment, beliefs on eco-driving, and driving behavior," Transportation Research Part A: Policy and Practice, Elsevier, vol. 105(C), pages 95-105.
    12. Han, Xuefeng & He, Hongwen & Wu, Jingda & Peng, Jiankun & Li, Yuecheng, 2019. "Energy management based on reinforcement learning with double deep Q-learning for a hybrid electric tracked vehicle," Applied Energy, Elsevier, vol. 254(C).
    13. Tom Fletcher & Kambiz Ebrahimi, 2020. "The Effect of Fuel Cell and Battery Size on Efficiency and Cell Lifetime for an L7e Fuel Cell Hybrid Vehicle," Energies, MDPI, vol. 13(22), pages 1-18, November.
    14. Zhang, Kaixuan & Ruan, Jiageng & Li, Tongyang & Cui, Hanghang & Wu, Changcheng, 2023. "The effects investigation of data-driven fitting cycle and deep deterministic policy gradient algorithm on energy management strategy of dual-motor electric bus," Energy, Elsevier, vol. 269(C).
    15. Ruan, Jiageng & Wu, Changcheng & Liang, Zhaowen & Liu, Kai & Li, Bin & Li, Weihan & Li, Tongyang, 2023. "The application of machine learning-based energy management strategy in a multi-mode plug-in hybrid electric vehicle, part II: Deep deterministic policy gradient algorithm design for electric mode," Energy, Elsevier, vol. 269(C).
    16. Li, Yuecheng & He, Hongwen & Khajepour, Amir & Wang, Hong & Peng, Jiankun, 2019. "Energy management for a power-split hybrid electric bus via deep reinforcement learning with terrain information," Applied Energy, Elsevier, vol. 255(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Liang, Zhaowen & Ruan, Jiageng & Wang, Zhenpo & Liu, Kai & Li, Bin, 2024. "Soft actor-critic-based EMS design for dual motor battery electric bus," Energy, Elsevier, vol. 288(C).
    2. Fengqi Zhang & Lihua Wang & Serdar Coskun & Hui Pang & Yahui Cui & Junqiang Xi, 2020. "Energy Management Strategies for Hybrid Electric Vehicles: Review, Classification, Comparison, and Outlook," Energies, MDPI, vol. 13(13), pages 1-35, June.
    3. Changqing Du & Shiyang Huang & Yuyao Jiang & Dongmei Wu & Yang Li, 2022. "Optimization of Energy Management Strategy for Fuel Cell Hybrid Electric Vehicles Based on Dynamic Programming," Energies, MDPI, vol. 15(12), pages 1-25, June.
    4. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    5. Penghui Qiang & Peng Wu & Tao Pan & Huaiquan Zang, 2021. "Real-Time Approximate Equivalent Consumption Minimization Strategy Based on the Single-Shaft Parallel Hybrid Powertrain," Energies, MDPI, vol. 14(23), pages 1-22, November.
    6. Chen, Jiaxin & Shu, Hong & Tang, Xiaolin & Liu, Teng & Wang, Weida, 2022. "Deep reinforcement learning-based multi-objective control of hybrid power system combined with road recognition under time-varying environment," Energy, Elsevier, vol. 239(PC).
    7. Yang, Ningkang & Ruan, Shumin & Han, Lijin & Liu, Hui & Guo, Lingxiong & Xiang, Changle, 2023. "Reinforcement learning-based real-time intelligent energy management for hybrid electric vehicles in a model predictive control framework," Energy, Elsevier, vol. 270(C).
    8. Alessia Musa & Pier Giuseppe Anselma & Giovanni Belingardi & Daniela Anna Misul, 2023. "Energy Management in Hybrid Electric Vehicles: A Q-Learning Solution for Enhanced Drivability and Energy Efficiency," Energies, MDPI, vol. 17(1), pages 1-20, December.
    9. Qi, Chunyang & Song, Chuanxue & Xiao, Feng & Song, Shixin, 2022. "Generalization ability of hybrid electric vehicle energy management strategy based on reinforcement learning method," Energy, Elsevier, vol. 250(C).
    10. Ju, Fei & Zhuang, Weichao & Wang, Liangmo & Zhang, Zhe, 2020. "Comparison of four-wheel-drive hybrid powertrain configurations," Energy, Elsevier, vol. 209(C).
    11. Yang, Dongpo & Liu, Tong & Song, Dafeng & Zhang, Xuanming & Zeng, Xiaohua, 2023. "A real time multi-objective optimization Guided-MPC strategy for power-split hybrid electric bus based on velocity prediction," Energy, Elsevier, vol. 276(C).
    12. Wang, Yue & Li, Keqiang & Zeng, Xiaohua & Gao, Bolin & Hong, Jichao, 2023. "Investigation of novel intelligent energy management strategies for connected HEB considering global planning of fixed-route information," Energy, Elsevier, vol. 263(PB).
    13. Xiao, Boyi & Yang, Weiwei & Wu, Jiamin & Walker, Paul D. & Zhang, Nong, 2022. "Energy management strategy via maximum entropy reinforcement learning for an extended range logistics vehicle," Energy, Elsevier, vol. 253(C).
    14. Li, Jie & Wu, Xiaodong & Xu, Min & Liu, Yonggang, 2022. "Deep reinforcement learning and reward shaping based eco-driving control for automated HEVs among signalized intersections," Energy, Elsevier, vol. 251(C).
    15. Geng, Wenran & Lou, Diming & Wang, Chen & Zhang, Tong, 2020. "A cascaded energy management optimization method of multimode power-split hybrid electric vehicles," Energy, Elsevier, vol. 199(C).
    16. Liu, Teng & Tan, Wenhao & Tang, Xiaolin & Zhang, Jinwei & Xing, Yang & Cao, Dongpu, 2021. "Driving conditions-driven energy management strategies for hybrid electric vehicles: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 151(C).
    17. Xiaodong Liu & Hongqiang Guo & Xingqun Cheng & Juan Du & Jian Ma, 2022. "A Robust Design of the Model-Free-Adaptive-Control-Based Energy Management for Plug-In Hybrid Electric Vehicle," Energies, MDPI, vol. 15(20), pages 1-24, October.
    18. Anselma, Pier Giuseppe, 2022. "Computationally efficient evaluation of fuel and electrical energy economy of plug-in hybrid electric vehicles with smooth driving constraints," Applied Energy, Elsevier, vol. 307(C).
    19. Miranda, Matheus H.R. & Silva, Fabrício L. & Lourenço, Maria A.M. & Eckert, Jony J. & Silva, Ludmila C.A., 2022. "Vehicle drivetrain and fuzzy controller optimization using a planar dynamics simulation based on a real-world driving cycle," Energy, Elsevier, vol. 257(C).
    20. Qi, Chunyang & Zhu, Yiwen & Song, Chuanxue & Yan, Guangfu & Xiao, Feng & Da wang, & Zhang, Xu & Cao, Jingwei & Song, Shixin, 2022. "Hierarchical reinforcement learning based energy management strategy for hybrid electric vehicle," Energy, Elsevier, vol. 238(PA).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:285:y:2023:i:c:s0360544223028438. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.