IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v313y2024ics0360544224038647.html
   My bibliography  Save this article

Adaptive deep reinforcement learning energy management for hybrid electric vehicles considering driving condition recognition

Author

Listed:
  • Zhang, Dehai
  • Li, Junhui
  • Guo, Ningyuan
  • Liu, Yonggang
  • Shen, Shiquan
  • Wei, Fuxing
  • Chen, Zheng
  • Zheng, Jia

Abstract

Adaptive energy management strategy (EMS) can adjust the underlying control scheme of hybrid electric vehicles (HEVs) according to different external conditions, thus maintaining high performance consistently. For this reason, this study designs a double-layer EMS for a power-split HEV, which combines deep reinforcement learning (DRL) with driving condition recognition (DCR) to achieve efficient operation of powertrain under different driving conditions. At the upper layer, the long-term dependencies in velocity sequences are captured through the sequential networks to enhance the accuracy of DCR. At the lower layer, multi-objective reward functions are formulated, and the optimal weight parameters are identified for different driving conditions. Furthermore, the deep deterministic policy gradient (DDPG) algorithm is employed to dynamically optimize power flow. The simulation experiments are conducted to verify the feasibility and the energy-saving effects of the proposed strategy. The results indicate that integrating variations in driving conditions into the energy allocation of HEVs can further improve the fuel economy and maintain battery health. Specifically, the method enhances the fuel economy by an average of 10.76 %, compared to a number of traditional benchmark strategies.

Suggested Citation

  • Zhang, Dehai & Li, Junhui & Guo, Ningyuan & Liu, Yonggang & Shen, Shiquan & Wei, Fuxing & Chen, Zheng & Zheng, Jia, 2024. "Adaptive deep reinforcement learning energy management for hybrid electric vehicles considering driving condition recognition," Energy, Elsevier, vol. 313(C).
    • Handle: RePEc:eee:energy:v:313:y:2024:i:c:s0360544224038647
    DOI: 10.1016/j.energy.2024.134086
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544224038647
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2024.134086?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lian, Renzong & Peng, Jiankun & Wu, Yuankai & Tan, Huachun & Zhang, Hailong, 2020. "Rule-interposing deep reinforcement learning based energy management strategy for power-split hybrid electric vehicle," Energy, Elsevier, vol. 197(C).
    2. Wang, Yichun & Zhang, Yuanzhi & Zhang, Caizhi & Zhou, Jiaming & Hu, Donghai & Yi, Fengyan & Fan, Zhixian & Zeng, Tao, 2023. "Genetic algorithm-based fuzzy optimization of energy management strategy for fuel cell vehicles considering driving cycles recognition," Energy, Elsevier, vol. 263(PF).
    3. Song, Ziyou & Hou, Jun & Xu, Shaobing & Ouyang, Minggao & Li, Jianqiu, 2017. "The influence of driving cycle characteristics on the integrated optimization of hybrid energy storage system for electric city buses," Energy, Elsevier, vol. 135(C), pages 91-100.
    4. Huang, Yin & Kang, Zehao & Mao, Xuping & Hu, Haoqin & Tan, Jiaqi & Xuan, Dongji, 2023. "Deep reinforcement learning based energymanagement strategy considering running costs and energy source aging for fuel cell hybrid electric vehicle," Energy, Elsevier, vol. 283(C).
    5. Guo, Lingxiong & Zhang, Xudong & Zou, Yuan & Han, Lijin & Du, Guodong & Guo, Ningyuan & Xiang, Changle, 2022. "Co-optimization strategy of unmanned hybrid electric tracked vehicle combining eco-driving and simultaneous energy management," Energy, Elsevier, vol. 246(C).
    6. Wang, Hanchen & Ye, Yiming & Zhang, Jiangfeng & Xu, Bin, 2023. "A comparative study of 13 deep reinforcement learning based energy management methods for a hybrid electric vehicle," Energy, Elsevier, vol. 266(C).
    7. Xu, Bin & Wang, Hanchen, 2023. "A comparative analysis of adaptive energy management for a hybrid electric vehicle via five driving condition recognition methods," Energy, Elsevier, vol. 269(C).
    8. Li, Xingshuo & Liu, Jinfu & Bai, Mingliang & Li, Jiajia & Li, Xianling & Yan, Peigang & Yu, Daren, 2021. "An LSTM based method for stage performance degradation early warning with consideration of time-series information," Energy, Elsevier, vol. 226(C).
    9. Xiong, Rui & Cao, Jiayi & Yu, Quanqing, 2018. "Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle," Applied Energy, Elsevier, vol. 211(C), pages 538-548.
    10. Li, Cheng & Xu, Xiangyang & Zhu, Helong & Gan, Jiongpeng & Chen, Zhige & Tang, Xiaolin, 2024. "Research on car-following control and energy management strategy of hybrid electric vehicles in connected scene," Energy, Elsevier, vol. 293(C).
    11. Xu, Bin & Rathod, Dhruvang & Zhang, Darui & Yebi, Adamu & Zhang, Xueyu & Li, Xiaoya & Filipi, Zoran, 2020. "Parametric study on reinforcement learning optimized energy management strategy for a hybrid electric vehicle," Applied Energy, Elsevier, vol. 259(C).
    12. Li, Jie & Wu, Xiaodong & Xu, Min & Liu, Yonggang, 2022. "Deep reinforcement learning and reward shaping based eco-driving control for automated HEVs among signalized intersections," Energy, Elsevier, vol. 251(C).
    13. Chen, Xu & Li, Mince & Chen, Zonghai, 2023. "Meta rule-based energy management strategy for battery/supercapacitor hybrid electric vehicles," Energy, Elsevier, vol. 285(C).
    14. Xu, Bin & Shi, Junzhe & Li, Sixu & Li, Huayi & Wang, Zhe, 2021. "Energy consumption and battery aging minimization using a Q-learning strategy for a battery/ultracapacitor electric vehicle," Energy, Elsevier, vol. 229(C).
    15. Guo, Ningyuan & Zhang, Wencan & Li, Junqiu & Chen, Zheng & Li, Jianwei & Sun, Chao, 2024. "Predictive energy management of fuel cell plug-in hybrid electric vehicles: A co-state boundaries-oriented PMP optimization approach," Applied Energy, Elsevier, vol. 362(C).
    16. Bao, Shuyue & Tang, Shifa & Sun, Ping & Wang, Tao, 2023. "LSTM-based energy management algorithm for a vehicle power-split hybrid powertrain," Energy, Elsevier, vol. 284(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lihua Ye & Zixing Zhang & Qinglong Zhao & Xu Zhao & Zhou He & Aiping Shi, 2025. "Research on Energy Management Strategies for Fuel Cell Hybrid Vehicles Based on Time Classification," Energies, MDPI, vol. 18(8), pages 1-29, April.
    2. Dawei Zhong & Bolan Liu & Liang Liu & Wenhao Fan & Jingxian Tang, 2025. "Artificial Intelligence Algorithms for Hybrid Electric Powertrain System Control: A Review," Energies, MDPI, vol. 18(8), pages 1-30, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhu, Tao & Wills, Richard G.A. & Lot, Roberto & Ruan, Haijun & Jiang, Zhihao, 2021. "Adaptive energy management of a battery-supercapacitor energy storage system for electric vehicles based on flexible perception and neural network fitting," Applied Energy, Elsevier, vol. 292(C).
    2. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    3. Shi, Dehua & Xu, Han & Wang, Shaohua & Hu, Jia & Chen, Long & Yin, Chunfang, 2024. "Deep reinforcement learning based adaptive energy management for plug-in hybrid electric vehicle with double deep Q-network," Energy, Elsevier, vol. 305(C).
    4. Liu, Weirong & Yao, Pengfei & Wu, Yue & Duan, Lijun & Li, Heng & Peng, Jun, 2025. "Imitation reinforcement learning energy management for electric vehicles with hybrid energy storage system," Applied Energy, Elsevier, vol. 378(PA).
    5. Dong, Peng & Zhao, Junwei & Liu, Xuewu & Wu, Jian & Xu, Xiangyang & Liu, Yanfang & Wang, Shuhan & Guo, Wei, 2022. "Practical application of energy management strategy for hybrid electric vehicles based on intelligent and connected technologies: Development stages, challenges, and future trends," Renewable and Sustainable Energy Reviews, Elsevier, vol. 170(C).
    6. Marouane Adnane & Ahmed Khoumsi & João Pedro F. Trovão, 2023. "Efficient Management of Energy Consumption of Electric Vehicles Using Machine Learning—A Systematic and Comprehensive Survey," Energies, MDPI, vol. 16(13), pages 1-39, June.
    7. Huang, Ruchen & He, Hongwen & Su, Qicong & Härtl, Martin & Jaensch, Malte, 2025. "Type- and task-crossing energy management for fuel cell vehicles with longevity consideration: A heterogeneous deep transfer reinforcement learning framework," Applied Energy, Elsevier, vol. 377(PC).
    8. Liu, Teng & Tan, Wenhao & Tang, Xiaolin & Zhang, Jinwei & Xing, Yang & Cao, Dongpu, 2021. "Driving conditions-driven energy management strategies for hybrid electric vehicles: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 151(C).
    9. Niu, Junyan & Zhuang, Weichao & Ye, Jianwei & Song, Ziyou & Yin, Guodong & Zhang, Yuanjian, 2022. "Optimal sizing and learning-based energy management strategy of NCR/LTO hybrid battery system for electric taxis," Energy, Elsevier, vol. 257(C).
    10. Ye, Yiming & Wang, Hanchen & Xu, Bin & Zhang, Jiangfeng, 2023. "An imitation learning-based energy management strategy for electric vehicles considering battery aging," Energy, Elsevier, vol. 283(C).
    11. Han, Lijin & You, Congwen & Yang, Ningkang & Liu, Hui & Chen, Ke & Xiang, Changle, 2024. "Adaptive real-time energy management strategy using heuristic search for off-road hybrid electric vehicles," Energy, Elsevier, vol. 304(C).
    12. Penghui Qiang & Peng Wu & Tao Pan & Huaiquan Zang, 2021. "Real-Time Approximate Equivalent Consumption Minimization Strategy Based on the Single-Shaft Parallel Hybrid Powertrain," Energies, MDPI, vol. 14(23), pages 1-22, November.
    13. Chen, Jiaxin & Shu, Hong & Tang, Xiaolin & Liu, Teng & Wang, Weida, 2022. "Deep reinforcement learning-based multi-objective control of hybrid power system combined with road recognition under time-varying environment," Energy, Elsevier, vol. 239(PC).
    14. Fengqi Zhang & Lihua Wang & Serdar Coskun & Hui Pang & Yahui Cui & Junqiang Xi, 2020. "Energy Management Strategies for Hybrid Electric Vehicles: Review, Classification, Comparison, and Outlook," Energies, MDPI, vol. 13(13), pages 1-35, June.
    15. Shi, Wenzhuo & Huangfu, Yigeng & Xu, Liangcai & Pang, Shengzhao, 2022. "Online energy management strategy considering fuel cell fault for multi-stack fuel cell hybrid vehicle based on multi-agent reinforcement learning," Applied Energy, Elsevier, vol. 328(C).
    16. Xiao, B. & Ruan, J. & Yang, W. & Walker, P.D. & Zhang, N., 2021. "A review of pivotal energy management strategies for extended range electric vehicles," Renewable and Sustainable Energy Reviews, Elsevier, vol. 149(C).
    17. Guo, Xiaokai & Yan, Xianguo & Chen, Zhi & Meng, Zhiyu, 2022. "Research on energy management strategy of heavy-duty fuel cell hybrid vehicles based on dueling-double-deep Q-network," Energy, Elsevier, vol. 260(C).
    18. Connor Scott & Mominul Ahsan & Alhussein Albarbar, 2021. "Machine Learning Based Vehicle to Grid Strategy for Improving the Energy Performance of Public Buildings," Sustainability, MDPI, vol. 13(7), pages 1-22, April.
    19. Alessia Musa & Pier Giuseppe Anselma & Giovanni Belingardi & Daniela Anna Misul, 2023. "Energy Management in Hybrid Electric Vehicles: A Q-Learning Solution for Enhanced Drivability and Energy Efficiency," Energies, MDPI, vol. 17(1), pages 1-20, December.
    20. Qi, Chunyang & Song, Chuanxue & Xiao, Feng & Song, Shixin, 2022. "Generalization ability of hybrid electric vehicle energy management strategy based on reinforcement learning method," Energy, Elsevier, vol. 250(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:313:y:2024:i:c:s0360544224038647. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.