IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v334y2025ics0360544225032360.html

A novel energy management strategy for hybrid electric vehicles using deep reinforcement incentive learning

Author

Listed:
  • Yazar, Ozan
  • Coskun, Serdar
  • Zhang, Fengqi
  • Li, Lin
  • Huang, Cong
  • Mei, Peng
  • Karimi, Hamid Reza

Abstract

Hybrid electric vehicles (HEVs) represent a significant step toward achieving fuel-efficient and environmentally friendly mobility. An effective energy management strategy (EMS) is needed to maximize the performance and energy efficiency of HEVs. However, since existing EMSs are generally developed utilizing rule-based or fixed structures, they often suffer from problems such as insufficient generalization capacity under variable driving conditions and exhibiting limited real-time applicability. Integrating deep reinforcement learning (DRL) algorithms into EMS design has attracted attention to overcome these limitations in recent years. DRL-based methods have a strong potential to develop more efficient and adaptive control strategies in HEVs. However, the proper design of reward functions representing multi-objective control goals and prioritizing the instantaneous objective remains a significant technical challenge, which can negatively affect the optimization performance during the learning process. We propose a deep reinforcement incentive learning (DRIL) algorithm to minimize fuel consumption and maintain battery charge sustainability under various driving conditions. To ignite the reinforcement learning agent to efficiently learn the optimal power distribution, we first propose an incentive reward function based on fuel consumption minimization and battery charge sustainability. Next, we introduce a novel soft actor-critic algorithm combined with incentive-based reward, forming the DRIL algorithm, which optimally balances exploration and exploitation for power allocation. Finally, the generalized advantage of the DRIL algorithm is evaluated through a human-in-the-loop (HIL) test-driving cycle, demonstrating its real-time applicability. The results of the experiments demonstrate that the proposed incentive mechanism can reduce fuel costs by 3.47%–3.04% compared to the existing DRL algorithms under the pre-training driving cycle and the HIL-obtained driving cycles, respectively.

Suggested Citation

  • Yazar, Ozan & Coskun, Serdar & Zhang, Fengqi & Li, Lin & Huang, Cong & Mei, Peng & Karimi, Hamid Reza, 2025. "A novel energy management strategy for hybrid electric vehicles using deep reinforcement incentive learning," Energy, Elsevier, vol. 334(C).
    • Handle: RePEc:eee:energy:v:334:y:2025:i:c:s0360544225032360
    DOI: 10.1016/j.energy.2025.137594
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544225032360
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2025.137594?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Du, Guodong & Zou, Yuan & Zhang, Xudong & Liu, Teng & Wu, Jinlong & He, Dingbo, 2020. "Deep reinforcement learning based energy management for a hybrid electric vehicle," Energy, Elsevier, vol. 201(C).
    2. Lian, Renzong & Peng, Jiankun & Wu, Yuankai & Tan, Huachun & Zhang, Hailong, 2020. "Rule-interposing deep reinforcement learning based energy management strategy for power-split hybrid electric vehicle," Energy, Elsevier, vol. 197(C).
    3. He, Hongwen & Meng, Xiangfei & Wang, Yong & Khajepour, Amir & An, Xiaowen & Wang, Renguang & Sun, Fengchun, 2024. "Deep reinforcement learning based energy management strategies for electrified vehicles: Recent advances and perspectives," Renewable and Sustainable Energy Reviews, Elsevier, vol. 192(C).
    4. Meng, Qinglin & He, Ying & Gao, Yuan & Hussain, Sheharyar & Lu, Jinghang & Guerrero, Josep M., 2025. "Bi-level four-stage optimization scheduling for Active Distribution Networks with Electric Vehicle integration using multi-mode dynamic pricing," Energy, Elsevier, vol. 327(C).
    5. Xiao Li & Hanchen Xu & Jinming Zhang & Hua-hua Chang, 2023. "Deep Reinforcement Learning for Adaptive Learning Systems," Journal of Educational and Behavioral Statistics, , vol. 48(2), pages 220-243, April.
    6. Emiliano Pipitone & Salvatore Caltabellotta & Leonardo Occhipinti, 2021. "A Life Cycle Environmental Impact Comparison between Traditional, Hybrid, and Electric Vehicles in the European Context," Sustainability, MDPI, vol. 13(19), pages 1-32, October.
    7. Wang, Hanchen & Ye, Yiming & Zhang, Jiangfeng & Xu, Bin, 2023. "A comparative study of 13 deep reinforcement learning based energy management methods for a hybrid electric vehicle," Energy, Elsevier, vol. 266(C).
    8. Han, Xuefeng & He, Hongwen & Wu, Jingda & Peng, Jiankun & Li, Yuecheng, 2019. "Energy management based on reinforcement learning with double deep Q-learning for a hybrid electric tracked vehicle," Applied Energy, Elsevier, vol. 254(C).
    9. Wen Wang & Beibei Li & Xueming Luo & Xiaoyi Wang, 2023. "Deep Reinforcement Learning for Sequential Targeting," Management Science, INFORMS, vol. 69(9), pages 5439-5460, September.
    10. Zhang, Dehai & Li, Junhui & Guo, Ningyuan & Liu, Yonggang & Shen, Shiquan & Wei, Fuxing & Chen, Zheng & Zheng, Jia, 2024. "Adaptive deep reinforcement learning energy management for hybrid electric vehicles considering driving condition recognition," Energy, Elsevier, vol. 313(C).
    11. Liang, Xuedong & Luo, Peng & Li, Xiaoyan & Wang, Xia & Shu, Lingli, 2023. "Crude oil price prediction using deep reinforcement learning," Resources Policy, Elsevier, vol. 81(C).
    12. Fengqi Zhang & Lihua Wang & Serdar Coskun & Hui Pang & Yahui Cui & Junqiang Xi, 2020. "Energy Management Strategies for Hybrid Electric Vehicles: Review, Classification, Comparison, and Outlook," Energies, MDPI, vol. 13(13), pages 1-35, June.
    13. Liu, Zemin Eitan & Li, Yong & Zhou, Quan & Shuai, Bin & Hua, Min & Xu, Hongming & Xu, Lubing & Tan, Guikun & Li, Yanfei, 2025. "Real-time energy management for HEV combining naturalistic driving data and deep reinforcement learning with high generalization," Applied Energy, Elsevier, vol. 377(PA).
    14. Zhou, Jianhao & Xue, Siwu & Xue, Yuan & Liao, Yuhui & Liu, Jun & Zhao, Wanzhong, 2021. "A novel energy management strategy of hybrid electric vehicle via an improved TD3 deep reinforcement learning," Energy, Elsevier, vol. 224(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wang, Yue & Li, Keqiang & Zeng, Xiaohua & Gao, Bolin & Hong, Jichao, 2023. "Investigation of novel intelligent energy management strategies for connected HEB considering global planning of fixed-route information," Energy, Elsevier, vol. 263(PB).
    2. Li, Xueliang & Liu, Yilong & Yan, Mei & Tian, Dayu & Yang, Shujun & Peng, Zengxiong, 2026. "Data-driven a convergence-enhanced fusion energy management strategy based on teacher agent guidance for hybrid electric vehicles," Applied Energy, Elsevier, vol. 404(C).
    3. Tang, Tianfeng & Peng, Qianlong & Shi, Qing & Peng, Qingguo & Zhao, Jin & Chen, Chaoyi & Wang, Guangwei, 2024. "Energy management of fuel cell hybrid electric bus in mountainous regions: A deep reinforcement learning approach considering terrain characteristics," Energy, Elsevier, vol. 311(C).
    4. Ju, Fei & Zhuang, Weichao & Wang, Liangmo & Zhang, Zhe, 2020. "Comparison of four-wheel-drive hybrid powertrain configurations," Energy, Elsevier, vol. 209(C).
    5. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    6. Miranda, Matheus H.R. & Silva, Fabrício L. & Lourenço, Maria A.M. & Eckert, Jony J. & Silva, Ludmila C.A., 2022. "Vehicle drivetrain and fuzzy controller optimization using a planar dynamics simulation based on a real-world driving cycle," Energy, Elsevier, vol. 257(C).
    7. Tan, Yingqi & Xu, Jingyi & Ma, Junyi & Li, Zirui & Chen, Huiyan & Xi, Junqiang & Liu, Haiou, 2024. "A transferable perception-guided EMS for series hybrid electric unmanned tracked vehicles," Energy, Elsevier, vol. 306(C).
    8. Liu, Zemin Eitan & Li, Yong & Zhou, Quan & Shuai, Bin & Hua, Min & Xu, Hongming & Xu, Lubing & Tan, Guikun & Li, Yanfei, 2025. "Real-time energy management for HEV combining naturalistic driving data and deep reinforcement learning with high generalization," Applied Energy, Elsevier, vol. 377(PA).
    9. Yang, Dongpo & Liu, Tong & Song, Dafeng & Zhang, Xuanming & Zeng, Xiaohua, 2023. "A real time multi-objective optimization Guided-MPC strategy for power-split hybrid electric bus based on velocity prediction," Energy, Elsevier, vol. 276(C).
    10. Huang, Ruchen & He, Hongwen & Su, Qicong, 2024. "Smart energy management for hybrid electric bus via improved soft actor-critic algorithm in a heuristic learning framework," Energy, Elsevier, vol. 309(C).
    11. Dawei Zhong & Bolan Liu & Liang Liu & Wenhao Fan & Jingxian Tang, 2025. "Artificial Intelligence Algorithms for Hybrid Electric Powertrain System Control: A Review," Energies, MDPI, vol. 18(8), pages 1-30, April.
    12. Yao, Yongming & Wang, Jie & Zhou, Zhicong & Li, Hang & Liu, Huiying & Li, Tianyu, 2023. "Grey Markov prediction-based hierarchical model predictive control energy management for fuel cell/battery hybrid unmanned aerial vehicles," Energy, Elsevier, vol. 262(PA).
    13. Dong, Peng & Zhao, Junwei & Liu, Xuewu & Wu, Jian & Xu, Xiangyang & Liu, Yanfang & Wang, Shuhan & Guo, Wei, 2022. "Practical application of energy management strategy for hybrid electric vehicles based on intelligent and connected technologies: Development stages, challenges, and future trends," Renewable and Sustainable Energy Reviews, Elsevier, vol. 170(C).
    14. Marouane Adnane & Ahmed Khoumsi & João Pedro F. Trovão, 2023. "Efficient Management of Energy Consumption of Electric Vehicles Using Machine Learning—A Systematic and Comprehensive Survey," Energies, MDPI, vol. 16(13), pages 1-39, June.
    15. Huang, Ruchen & He, Hongwen, 2025. "UpdatingEMS: An online updating framework for deep reinforcement learning-based energy management of fuel cell hybrid electric bus with integrated transfer learning," Applied Energy, Elsevier, vol. 402(PA).
    16. Kunyu Wang & Rong Yang & Yongjian Zhou & Wei Huang & Song Zhang, 2022. "Design and Improvement of SD3-Based Energy Management Strategy for a Hybrid Electric Urban Bus," Energies, MDPI, vol. 15(16), pages 1-21, August.
    17. Alessia Musa & Pier Giuseppe Anselma & Giovanni Belingardi & Daniela Anna Misul, 2023. "Energy Management in Hybrid Electric Vehicles: A Q-Learning Solution for Enhanced Drivability and Energy Efficiency," Energies, MDPI, vol. 17(1), pages 1-20, December.
    18. Liu, Teng & Tan, Wenhao & Tang, Xiaolin & Zhang, Jinwei & Xing, Yang & Cao, Dongpu, 2021. "Driving conditions-driven energy management strategies for hybrid electric vehicles: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 151(C).
    19. Peng, Xianlun & Li, Fangfei & Liu, Yang, 2026. "Reinforcement learning-based optimal false data injection attack against control signals in probabilistic Boolean control networks," Applied Mathematics and Computation, Elsevier, vol. 519(C).
    20. Li, Jianwei & Liu, Jie & Yang, Qingqing & Wang, Tianci & He, Hongwen & Wang, Hanxiao & Sun, Fengchun, 2025. "Reinforcement learning based energy management for fuel cell hybrid electric vehicles: A comprehensive review on decision process reformulation and strategy implementation," Renewable and Sustainable Energy Reviews, Elsevier, vol. 213(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:334:y:2025:i:c:s0360544225032360. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.