IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v15y2023i16p12488-d1218886.html
   My bibliography  Save this article

Research on Energy Management of Hydrogen Fuel Cell Bus Based on Deep Reinforcement Learning Considering Velocity Control

Author

Listed:
  • Yang Shen

    (School of Automotive Engineering, Shandong Jiaotong University, Jinan 250357, China)

  • Jiaming Zhou

    (School of Intelligent Manufacturing, Weifang University of Science and Technology, Weifang 262700, China)

  • Jinming Zhang

    (School of Intelligent Manufacturing, Weifang University of Science and Technology, Weifang 262700, China)

  • Fengyan Yi

    (School of Automotive Engineering, Shandong Jiaotong University, Jinan 250357, China)

  • Guofeng Wang

    (School of Automotive Engineering, Shandong Jiaotong University, Jinan 250357, China)

  • Chaofeng Pan

    (School of Automotive Engineering, Jiangsu University, Zhenjiang 212013, China)

  • Wei Guo

    (School of Automotive Engineering, Shandong Jiaotong University, Jinan 250357, China)

  • Xing Shu

    (School of Automotive Engineering, Shandong Jiaotong University, Jinan 250357, China)

Abstract

In the vehicle-to-everything scenario, the fuel cell bus can accurately obtain the surrounding traffic information, and quickly optimize the energy management problem while controlling its own safe and efficient driving. This paper proposes an energy management strategy (EMS) that considers speed control based on deep reinforcement learning (DRL) in complex traffic scenarios. Using SUMO simulation software (Version 1.15.0), a two-lane urban expressway is designed as a traffic scenario, and a hydrogen fuel cell bus speed control and energy management system is designed through the soft actor–critic (SAC) algorithm to effectively reduce the equivalent hydrogen consumption and fuel cell output power fluctuation while ensuring the safe, efficient and smooth driving of the vehicle. Compared with the SUMO–IDM car-following model, the average speed of vehicles is kept the same, and the average acceleration and acceleration change value decrease by 10.22% and 11.57% respectively. Compared with deep deterministic policy gradient (DDPG), the average speed is increased by 1.18%, and the average acceleration and acceleration change value are decreased by 4.82% and 5.31% respectively. In terms of energy management, the hydrogen consumption of SAC–OPT-based energy management strategy reaches 95.52% of that of the DP algorithm, and the fluctuation range is reduced by 32.65%. Compared with SAC strategy, the fluctuation amplitude is reduced by 15.29%, which effectively improves the durability of fuel cells.

Suggested Citation

  • Yang Shen & Jiaming Zhou & Jinming Zhang & Fengyan Yi & Guofeng Wang & Chaofeng Pan & Wei Guo & Xing Shu, 2023. "Research on Energy Management of Hydrogen Fuel Cell Bus Based on Deep Reinforcement Learning Considering Velocity Control," Sustainability, MDPI, vol. 15(16), pages 1-19, August.
  • Handle: RePEc:gam:jsusta:v:15:y:2023:i:16:p:12488-:d:1218886
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/15/16/12488/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2071-1050/15/16/12488/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jiang, Hongliang & Xu, Liangfei & Li, Jianqiu & Hu, Zunyan & Ouyang, Minggao, 2019. "Energy management and component sizing for a fuel cell/battery/supercapacitor hybrid powertrain based on two-dimensional optimization algorithms," Energy, Elsevier, vol. 177(C), pages 386-396.
    2. Wang, Yichun & Zhang, Yuanzhi & Zhang, Caizhi & Zhou, Jiaming & Hu, Donghai & Yi, Fengyan & Fan, Zhixian & Zeng, Tao, 2023. "Genetic algorithm-based fuzzy optimization of energy management strategy for fuel cell vehicles considering driving cycles recognition," Energy, Elsevier, vol. 263(PF).
    3. Hou, Cong & Ouyang, Minggao & Xu, Liangfei & Wang, Hewu, 2014. "Approximate Pontryagin’s minimum principle applied to the energy management of plug-in hybrid electric vehicles," Applied Energy, Elsevier, vol. 115(C), pages 174-189.
    4. Min, Dehao & Song, Zhen & Chen, Huicui & Wang, Tianxiang & Zhang, Tong, 2022. "Genetic algorithm optimized neural network based fuel cell hybrid electric vehicle energy management strategy under start-stop condition," Applied Energy, Elsevier, vol. 306(PB).
    5. Škugor, Branimir & Deur, Joško, 2015. "Dynamic programming-based optimisation of charging an electric vehicle fleet system represented by an aggregate battery model," Energy, Elsevier, vol. 92(P3), pages 456-465.
    6. Tang, Xiaolin & Zhou, Haitao & Wang, Feng & Wang, Weida & Lin, Xianke, 2022. "Longevity-conscious energy management strategy of fuel cell hybrid electric Vehicle Based on deep reinforcement learning," Energy, Elsevier, vol. 238(PA).
    7. Lu, Dagang & Yi, Fengyan & Hu, Donghai & Li, Jianwei & Yang, Qingqing & Wang, Jing, 2023. "Online optimization of energy management strategy for FCV control parameters considering dual power source lifespan decay synergy," Applied Energy, Elsevier, vol. 348(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mubashir Rasool & Muhammad Adil Khan & Runmin Zou, 2023. "A Comprehensive Analysis of Online and Offline Energy Management Approaches for Optimal Performance of Fuel Cell Hybrid Electric Vehicles," Energies, MDPI, vol. 16(8), pages 1-33, April.
    2. Xun, Qian & Murgovski, Nikolce & Liu, Yujing, 2022. "Chance-constrained robust co-design optimization for fuel cell hybrid electric trucks," Applied Energy, Elsevier, vol. 320(C).
    3. Halder, Pobitra & Babaie, Meisam & Salek, Farhad & Shah, Kalpit & Stevanovic, Svetlana & Bodisco, Timothy A. & Zare, Ali, 2024. "Performance, emissions and economic analyses of hydrogen fuel cell vehicles," Renewable and Sustainable Energy Reviews, Elsevier, vol. 199(C).
    4. Peng, Hujun & Chen, Zhu & Li, Jianxiang & Deng, Kai & Dirkes, Steffen & Gottschalk, Jonas & Ünlübayir, Cem & Thul, Andreas & Löwenstein, Lars & Pischinger, Stefan & Hameyer, Kay, 2021. "Offline optimal energy management strategies considering high dynamics in batteries and constraints on fuel cell system power rate: From analytical derivation to validation on test bench," Applied Energy, Elsevier, vol. 282(PA).
    5. Aissa Benhammou & Hamza Tedjini & Mohammed Amine Hartani & Rania M. Ghoniem & Ali Alahmer, 2023. "Accurate and Efficient Energy Management System of Fuel Cell/Battery/Supercapacitor/AC and DC Generators Hybrid Electric Vehicles," Sustainability, MDPI, vol. 15(13), pages 1-27, June.
    6. Liu, Teng & Wang, Bo & Yang, Chenglang, 2018. "Online Markov Chain-based energy management for a hybrid tracked vehicle with speedy Q-learning," Energy, Elsevier, vol. 160(C), pages 544-555.
    7. Jin, Ruiyang & Zhou, Yuke & Lu, Chao & Song, Jie, 2022. "Deep reinforcement learning-based strategy for charging station participating in demand response," Applied Energy, Elsevier, vol. 328(C).
    8. Østergaard, P.A. & Lund, H. & Thellufsen, J.Z. & Sorknæs, P. & Mathiesen, B.V., 2022. "Review and validation of EnergyPLAN," Renewable and Sustainable Energy Reviews, Elsevier, vol. 168(C).
    9. Bizon, Nicu, 2019. "Real-time optimization strategies of Fuel Cell Hybrid Power Systems based on Load-following control: A new strategy, and a comparative study of topologies and fuel economy obtained," Applied Energy, Elsevier, vol. 241(C), pages 444-460.
    10. Peng, Fei & Zhao, Yuanzhe & Li, Xiaopeng & Liu, Zhixiang & Chen, Weirong & Liu, Yang & Zhou, Donghua, 2017. "Development of master-slave energy management strategy based on fuzzy logic hysteresis state machine and differential power processing compensation for a PEMFC-LIB-SC hybrid tramway," Applied Energy, Elsevier, vol. 206(C), pages 346-363.
    11. Huang, Ruchen & He, Hongwen & Gao, Miaojue, 2023. "Training-efficient and cost-optimal energy management for fuel cell hybrid electric bus based on a novel distributed deep reinforcement learning framework," Applied Energy, Elsevier, vol. 346(C).
    12. Bảo-Huy Nguyễn & João Pedro F. Trovão & Ronan German & Alain Bouscayrol, 2020. "Real-Time Energy Management of Parallel Hybrid Electric Vehicles Using Linear Quadratic Regulation," Energies, MDPI, vol. 13(21), pages 1-19, October.
    13. Angel Recalde & Ricardo Cajo & Washington Velasquez & Manuel S. Alvarez-Alvarado, 2024. "Machine Learning and Optimization in Energy Management Systems for Plug-In Hybrid Electric Vehicles: A Comprehensive Review," Energies, MDPI, vol. 17(13), pages 1-39, June.
    14. Zhang, Zhonghao & Guo, Mengdi & Yu, Zhonghao & Yao, Siyue & Wang, Jin & Qiu, Diankai & Peng, Linfa, 2022. "A novel cooperative design with optimized flow field on bipolar plates and hybrid wettability gas diffusion layer for proton exchange membrane unitized regenerative fuel cell," Energy, Elsevier, vol. 239(PD).
    15. Chen, Zheng & Hu, Hengjie & Wu, Yitao & Zhang, Yuanjian & Li, Guang & Liu, Yonggang, 2020. "Stochastic model predictive control for energy management of power-split plug-in hybrid electric vehicles based on reinforcement learning," Energy, Elsevier, vol. 211(C).
    16. Afzal, Asif & Buradi, Abdulrajak & Jilte, Ravindra & Shaik, Saboor & Kaladgi, Abdul Razak & Arıcı, Muslum & Lee, Chew Tin & Nižetić, Sandro, 2023. "Optimizing the thermal performance of solar energy devices using meta-heuristic algorithms: A critical review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 173(C).
    17. Anselma, Pier Giuseppe, 2022. "Computationally efficient evaluation of fuel and electrical energy economy of plug-in hybrid electric vehicles with smooth driving constraints," Applied Energy, Elsevier, vol. 307(C).
    18. Chaofeng Pan & Yanyan Liang & Long Chen & Liao Chen, 2019. "Optimal Control for Hybrid Energy Storage Electric Vehicle to Achieve Energy Saving Using Dynamic Programming Approach," Energies, MDPI, vol. 12(4), pages 1-19, February.
    19. Ju, Fei & Zhuang, Weichao & Wang, Liangmo & Zhang, Zhe, 2020. "Comparison of four-wheel-drive hybrid powertrain configurations," Energy, Elsevier, vol. 209(C).
    20. Jiang, Hongliang & Xu, Liangfei & Li, Jianqiu & Hu, Zunyan & Ouyang, Minggao, 2019. "Energy management and component sizing for a fuel cell/battery/supercapacitor hybrid powertrain based on two-dimensional optimization algorithms," Energy, Elsevier, vol. 177(C), pages 386-396.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:15:y:2023:i:16:p:12488-:d:1218886. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.