IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v238y2022ipas0360544221018417.html
   My bibliography  Save this article

Longevity-conscious energy management strategy of fuel cell hybrid electric Vehicle Based on deep reinforcement learning

Author

Listed:
  • Tang, Xiaolin
  • Zhou, Haitao
  • Wang, Feng
  • Wang, Weida
  • Lin, Xianke

Abstract

Deep reinforcement learning-based energy management strategy play an essential role in improving fuel economy and extending fuel cell lifetime for fuel cell hybrid electric vehicles. In this work, the traditional Deep Q-Network is compared with the Deep Q-Network with prioritized experience replay. Furthermore, the Deep Q-Network with prioritized experience replay is designed for energy management strategy to minimize hydrogen consumption and compared with the dynamic programming. Moreover, the fuel cell system degradation is incorporated into the objective function, and a balance between fuel economy and fuel cell system degradation is achieved by adjusting the degradation weight and the hydrogen consumption weight. Finally, the combined driving cycle is selected to further verify the effectiveness of the proposed strategy in unfamiliar driving environments and untrained situations. The training results under UDDS show that the fuel economy of the EMS decreases by 0.53 % when fuel cell system degradation is considered, reaching 88.73 % of the DP-based EMS in the UDDS, and the degradation of fuel cell system is effectively suppressed. At the same time, the computational efficiency is improved by more than 70 % compared to the DP-based strategy.

Suggested Citation

  • Tang, Xiaolin & Zhou, Haitao & Wang, Feng & Wang, Weida & Lin, Xianke, 2022. "Longevity-conscious energy management strategy of fuel cell hybrid electric Vehicle Based on deep reinforcement learning," Energy, Elsevier, vol. 238(PA).
  • Handle: RePEc:eee:energy:v:238:y:2022:i:pa:s0360544221018417
    DOI: 10.1016/j.energy.2021.121593
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544221018417
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.energy.2021.121593?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lian, Renzong & Peng, Jiankun & Wu, Yuankai & Tan, Huachun & Zhang, Hailong, 2020. "Rule-interposing deep reinforcement learning based energy management strategy for power-split hybrid electric vehicle," Energy, Elsevier, vol. 197(C).
    2. Sulaiman, N. & Hannan, M.A. & Mohamed, A. & Ker, P.J. & Majlan, E.H. & Wan Daud, W.R., 2018. "Optimization of energy management system for fuel-cell hybrid electric vehicles: Issues and recommendations," Applied Energy, Elsevier, vol. 228(C), pages 2061-2079.
    3. Abd Alla, Sara & Bianco, Vincenzo & Tagliafico, Luca A. & Scarpa, Federico, 2021. "Pathways to electric mobility integration in the Italian automotive sector," Energy, Elsevier, vol. 221(C).
    4. Du, Guodong & Zou, Yuan & Zhang, Xudong & Liu, Teng & Wu, Jinlong & He, Dingbo, 2020. "Deep reinforcement learning based energy management for a hybrid electric vehicle," Energy, Elsevier, vol. 201(C).
    5. Chen, Huicui & Pei, Pucheng & Song, Mancun, 2015. "Lifetime prediction and the economic lifetime of Proton Exchange Membrane fuel cells," Applied Energy, Elsevier, vol. 142(C), pages 154-163.
    6. Zhang, Pei & Yan, Fuwu & Du, Changqing, 2015. "A comprehensive analysis of energy management strategies for hybrid electric vehicles based on bibliometrics," Renewable and Sustainable Energy Reviews, Elsevier, vol. 48(C), pages 88-104.
    7. Baroutaji, Ahmad & Wilberforce, Tabbi & Ramadan, Mohamad & Olabi, Abdul Ghani, 2019. "Comprehensive investigation on hydrogen and fuel cell technology in the aviation and aerospace sectors," Renewable and Sustainable Energy Reviews, Elsevier, vol. 106(C), pages 31-40.
    8. Xie, Shaobo & Hu, Xiaosong & Qi, Shanwei & Tang, Xiaolin & Lang, Kun & Xin, Zongke & Brighton, James, 2019. "Model predictive energy management for plug-in hybrid electric vehicles considering optimal battery depth of discharge," Energy, Elsevier, vol. 173(C), pages 667-678.
    9. Zhou, Yang & Ravey, Alexandre & Péra, Marie-Cecile, 2020. "Multi-mode predictive energy management for fuel cell hybrid electric vehicles using Markov driving pattern recognizer," Applied Energy, Elsevier, vol. 258(C).
    10. Hu, Xiaosong & Zhang, Xiaoqian & Tang, Xiaolin & Lin, Xianke, 2020. "Model predictive control of hybrid electric vehicles for fuel economy, emission reductions, and inter-vehicle safety in car-following scenarios," Energy, Elsevier, vol. 196(C).
    11. Bizon, Nicu & Thounthong, Phatiphat, 2018. "Real-time strategies to optimize the fueling of the fuel cell hybrid power source: A review of issues, challenges and a new approach," Renewable and Sustainable Energy Reviews, Elsevier, vol. 91(C), pages 1089-1102.
    12. Wu, Jingda & He, Hongwen & Peng, Jiankun & Li, Yuecheng & Li, Zhanjiang, 2018. "Continuous reinforcement learning of energy management with deep Q network for a power split hybrid electric bus," Applied Energy, Elsevier, vol. 222(C), pages 799-811.
    13. Das, Himadry Shekhar & Tan, Chee Wei & Yatim, A.H.M., 2017. "Fuel cell hybrid electric vehicles: A review on power conditioning units and topologies," Renewable and Sustainable Energy Reviews, Elsevier, vol. 76(C), pages 268-291.
    14. Qin, Yechen & Tang, Xiaolin & Jia, Tong & Duan, Ziwen & Zhang, Jieming & Li, Yinong & Zheng, Ling, 2020. "Noise and vibration suppression in hybrid electric vehicles: State of the art and challenges," Renewable and Sustainable Energy Reviews, Elsevier, vol. 124(C).
    15. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    16. Yongliang Zheng & Feng He & Xinze Shen & Xuesheng Jiang, 2020. "Energy Control Strategy of Fuel Cell Hybrid Electric Vehicle Based on Working Conditions Identification by Least Square Support Vector Machine," Energies, MDPI, vol. 13(2), pages 1-18, January.
    17. Jinquan, Guo & Hongwen, He & Jianwei, Li & Qingwu, Liu, 2021. "Real-time energy management of fuel cell hybrid electric buses: Fuel cell engines friendly intersection speed planning," Energy, Elsevier, vol. 226(C).
    18. Olabi, A.G. & Wilberforce, Tabbi & Abdelkareem, Mohammad Ali, 2021. "Fuel cell application in the automotive industry and future perspective," Energy, Elsevier, vol. 214(C).
    19. Zhou, Jianhao & Xue, Siwu & Xue, Yuan & Liao, Yuhui & Liu, Jun & Zhao, Wanzhong, 2021. "A novel energy management strategy of hybrid electric vehicle via an improved TD3 deep reinforcement learning," Energy, Elsevier, vol. 224(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Huang, Ruchen & He, Hongwen & Zhao, Xuyang & Wang, Yunlong & Li, Menglin, 2022. "Battery health-aware and naturalistic data-driven energy management for hybrid electric bus based on TD3 deep reinforcement learning algorithm," Applied Energy, Elsevier, vol. 321(C).
    2. Li, Renzheng & Hong, Jichao & Zhang, Huaqin & Chen, Xinbo, 2022. "Data-driven battery state of health estimation based on interval capacity for real-world electric vehicles," Energy, Elsevier, vol. 257(C).
    3. Lin, Xinyou & Xu, Xinhao & Wang, Zhaorui, 2022. "Deep Q-learning network based trip pattern adaptive battery longevity-conscious strategy of plug-in fuel cell hybrid electric vehicle," Applied Energy, Elsevier, vol. 321(C).
    4. Mojgan Fayyazi & Paramjotsingh Sardar & Sumit Infent Thomas & Roonak Daghigh & Ali Jamali & Thomas Esch & Hans Kemper & Reza Langari & Hamid Khayyam, 2023. "Artificial Intelligence/Machine Learning in Energy Management Systems, Control, and Optimization of Hydrogen Fuel Cell Vehicles," Sustainability, MDPI, vol. 15(6), pages 1-38, March.
    5. Huang, Ruchen & He, Hongwen & Gao, Miaojue, 2023. "Training-efficient and cost-optimal energy management for fuel cell hybrid electric bus based on a novel distributed deep reinforcement learning framework," Applied Energy, Elsevier, vol. 346(C).
    6. Xun, Qian & Murgovski, Nikolce & Liu, Yujing, 2022. "Chance-constrained robust co-design optimization for fuel cell hybrid electric trucks," Applied Energy, Elsevier, vol. 320(C).
    7. Ong, Samuel & Al-Othman, Amani & Tawalbeh, Muhammad, 2023. "Emerging technologies in prognostics for fuel cells including direct hydrocarbon fuel cells," Energy, Elsevier, vol. 277(C).
    8. Lu, Dagang & Yi, Fengyan & Hu, Donghai & Li, Jianwei & Yang, Qingqing & Wang, Jing, 2023. "Online optimization of energy management strategy for FCV control parameters considering dual power source lifespan decay synergy," Applied Energy, Elsevier, vol. 348(C).
    9. Mokesioluwa Fanoro & Mladen Božanić & Saurabh Sinha, 2022. "A Review of the Impact of Battery Degradation on Energy Management Systems with a Special Emphasis on Electric Vehicles," Energies, MDPI, vol. 15(16), pages 1-29, August.
    10. Xin Peng & Hui Chen & Cong Guan, 2023. "Energy Management Optimization of Fuel Cell Hybrid Ship Based on Particle Swarm Optimization Algorithm," Energies, MDPI, vol. 16(3), pages 1-15, January.
    11. Marouane Adnane & Ahmed Khoumsi & João Pedro F. Trovão, 2023. "Efficient Management of Energy Consumption of Electric Vehicles Using Machine Learning—A Systematic and Comprehensive Survey," Energies, MDPI, vol. 16(13), pages 1-39, June.
    12. Mubashir Rasool & Muhammad Adil Khan & Runmin Zou, 2023. "A Comprehensive Analysis of Online and Offline Energy Management Approaches for Optimal Performance of Fuel Cell Hybrid Electric Vehicles," Energies, MDPI, vol. 16(8), pages 1-33, April.
    13. Yang Gao & Changhong Liu & Yuan Liang & Sadegh Kouhestani Hamed & Fuwei Wang & Bo Bi, 2022. "Minimizing Energy Consumption and Powertrain Cost of Fuel Cell Hybrid Vehicles with Consideration of Different Driving Cycles and SOC Ranges," Energies, MDPI, vol. 15(17), pages 1-12, August.
    14. Yang Shen & Jiaming Zhou & Jinming Zhang & Fengyan Yi & Guofeng Wang & Chaofeng Pan & Wei Guo & Xing Shu, 2023. "Research on Energy Management of Hydrogen Fuel Cell Bus Based on Deep Reinforcement Learning Considering Velocity Control," Sustainability, MDPI, vol. 15(16), pages 1-19, August.
    15. Wang, Yue & Li, Keqiang & Zeng, Xiaohua & Gao, Bolin & Hong, Jichao, 2023. "Investigation of novel intelligent energy management strategies for connected HEB considering global planning of fixed-route information," Energy, Elsevier, vol. 263(PB).
    16. Yanwei Liu & Jiansheng Liang & Jiaqing Song & Jie Ye, 2022. "Research on Energy Management Strategy of Fuel Cell Vehicle Based on Multi-Dimensional Dynamic Programming," Energies, MDPI, vol. 15(14), pages 1-20, July.
    17. Yin, Linfei & Li, Yu, 2022. "Hybrid multi-agent emotional deep Q network for generation control of multi-area integrated energy systems," Applied Energy, Elsevier, vol. 324(C).
    18. Hossein Pourrahmani & Majid Siavashi & Adel Yavarinasab & Mardit Matian & Nazanin Chitgar & Ligang Wang & Jan Van herle, 2022. "A Review on the Long-Term Performance of Proton Exchange Membrane Fuel Cells: From Degradation Modeling to the Effects of Bipolar Plates, Sealings, and Contaminants," Energies, MDPI, vol. 15(14), pages 1-30, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Liu, Teng & Tan, Wenhao & Tang, Xiaolin & Zhang, Jinwei & Xing, Yang & Cao, Dongpu, 2021. "Driving conditions-driven energy management strategies for hybrid electric vehicles: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 151(C).
    2. Qi, Chunyang & Zhu, Yiwen & Song, Chuanxue & Yan, Guangfu & Xiao, Feng & Da wang, & Zhang, Xu & Cao, Jingwei & Song, Shixin, 2022. "Hierarchical reinforcement learning based energy management strategy for hybrid electric vehicle," Energy, Elsevier, vol. 238(PA).
    3. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    4. Chen, Jiaxin & Shu, Hong & Tang, Xiaolin & Liu, Teng & Wang, Weida, 2022. "Deep reinforcement learning-based multi-objective control of hybrid power system combined with road recognition under time-varying environment," Energy, Elsevier, vol. 239(PC).
    5. Zhengyu Yao & Hwan-Sik Yoon & Yang-Ki Hong, 2023. "Control of Hybrid Electric Vehicle Powertrain Using Offline-Online Hybrid Reinforcement Learning," Energies, MDPI, vol. 16(2), pages 1-18, January.
    6. Wang, Yue & Li, Keqiang & Zeng, Xiaohua & Gao, Bolin & Hong, Jichao, 2023. "Investigation of novel intelligent energy management strategies for connected HEB considering global planning of fixed-route information," Energy, Elsevier, vol. 263(PB).
    7. Dong, Peng & Zhao, Junwei & Liu, Xuewu & Wu, Jian & Xu, Xiangyang & Liu, Yanfang & Wang, Shuhan & Guo, Wei, 2022. "Practical application of energy management strategy for hybrid electric vehicles based on intelligent and connected technologies: Development stages, challenges, and future trends," Renewable and Sustainable Energy Reviews, Elsevier, vol. 170(C).
    8. Guo, Ningyuan & Zhang, Xudong & Zou, Yuan & Guo, Lingxiong & Du, Guodong, 2021. "Real-time predictive energy management of plug-in hybrid electric vehicles for coordination of fuel economy and battery degradation," Energy, Elsevier, vol. 214(C).
    9. Kunyu Wang & Rong Yang & Yongjian Zhou & Wei Huang & Song Zhang, 2022. "Design and Improvement of SD3-Based Energy Management Strategy for a Hybrid Electric Urban Bus," Energies, MDPI, vol. 15(16), pages 1-21, August.
    10. Vamsi Krishna Reddy, Aala Kalananda & Venkata Lakshmi Narayana, Komanapalli, 2022. "Meta-heuristics optimization in electric vehicles -an extensive review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 160(C).
    11. Balali, Yasaman & Stegen, Sascha, 2021. "Review of energy storage systems for vehicles based on technology, environmental impacts, and costs," Renewable and Sustainable Energy Reviews, Elsevier, vol. 135(C).
    12. Jinquan, Guo & Hongwen, He & Jianwei, Li & Qingwu, Liu, 2022. "Driving information process system-based real-time energy management for the fuel cell bus to minimize fuel cell engine aging and energy consumption," Energy, Elsevier, vol. 248(C).
    13. Bizon, Nicu, 2019. "Real-time optimization strategies of Fuel Cell Hybrid Power Systems based on Load-following control: A new strategy, and a comparative study of topologies and fuel economy obtained," Applied Energy, Elsevier, vol. 241(C), pages 444-460.
    14. Zhou, Jianhao & Liu, Jun & Xue, Yuan & Liao, Yuhui, 2022. "Total travel costs minimization strategy of a dual-stack fuel cell logistics truck enhanced with artificial potential field and deep reinforcement learning," Energy, Elsevier, vol. 239(PA).
    15. Penghui Qiang & Peng Wu & Tao Pan & Huaiquan Zang, 2021. "Real-Time Approximate Equivalent Consumption Minimization Strategy Based on the Single-Shaft Parallel Hybrid Powertrain," Energies, MDPI, vol. 14(23), pages 1-22, November.
    16. Zhuang, Weichao & Li (Eben), Shengbo & Zhang, Xiaowu & Kum, Dongsuk & Song, Ziyou & Yin, Guodong & Ju, Fei, 2020. "A survey of powertrain configuration studies on hybrid electric vehicles," Applied Energy, Elsevier, vol. 262(C).
    17. Huang, Ruchen & He, Hongwen & Zhao, Xuyang & Wang, Yunlong & Li, Menglin, 2022. "Battery health-aware and naturalistic data-driven energy management for hybrid electric bus based on TD3 deep reinforcement learning algorithm," Applied Energy, Elsevier, vol. 321(C).
    18. Zhou, Jianhao & Xue, Yuan & Xu, Da & Li, Chaoxiong & Zhao, Wanzhong, 2022. "Self-learning energy management strategy for hybrid electric vehicle via curiosity-inspired asynchronous deep reinforcement learning," Energy, Elsevier, vol. 242(C).
    19. Ma, Shuai & Lin, Meng & Lin, Tzu-En & Lan, Tian & Liao, Xun & Maréchal, François & Van herle, Jan & Yang, Yongping & Dong, Changqing & Wang, Ligang, 2021. "Fuel cell-battery hybrid systems for mobility and off-grid applications: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 135(C).
    20. Yao, Yongming & Wang, Jie & Zhou, Zhicong & Li, Hang & Liu, Huiying & Li, Tianyu, 2023. "Grey Markov prediction-based hierarchical model predictive control energy management for fuel cell/battery hybrid unmanned aerial vehicles," Energy, Elsevier, vol. 262(PA).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:238:y:2022:i:pa:s0360544221018417. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.