IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v381y2025ics0306261924025224.html
   My bibliography  Save this article

Enhancing data-driven energy management strategy via digital expert guidance for electrified vehicles

Author

Listed:
  • Hu, Dong
  • Huang, Chao
  • Wu, Jingda
  • Wei, Henglai
  • Pi, Dawei

Abstract

This study addresses data efficiency and reliability issues in reinforcement learning (RL)-based energy management strategies (EMS) for hybrid electric vehicles (HEVs). A novel expert-guided RL (EGRL) paradigm is proposed, combining deep ensemble methods with a digital expert model (DEM) for real-time EMS intervention across various scenarios. DEM, trained via domain adversarially invariant meta-learning (DAIML), adapts to different driving conditions. An intervention mechanism, based on uncertainty evaluation in the deep ensemble, allows DEM to guide and supervise RL training, ensuring reliability. The EMS optimizes energy consumption, battery health, and electricity maintenance for the range-extended electric bus (REEB) system. Simulation results show the paradigm significantly improves energy management, nearing optimal performance and surpassing traditional RL methods. EGRL achieves an average 15.8% improvement in economic benefit across all test cycles. This research offers an innovative solution for EMS and has broad potential for other automation applications.

Suggested Citation

  • Hu, Dong & Huang, Chao & Wu, Jingda & Wei, Henglai & Pi, Dawei, 2025. "Enhancing data-driven energy management strategy via digital expert guidance for electrified vehicles," Applied Energy, Elsevier, vol. 381(C).
    • Handle: RePEc:eee:appene:v:381:y:2025:i:c:s0306261924025224
    DOI: 10.1016/j.apenergy.2024.125138
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261924025224
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2024.125138?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. He, Hongwen & Wang, Yunlong & Han, Ruoyan & Han, Mo & Bai, Yunfei & Liu, Qingwu, 2021. "An improved MPC-based energy management strategy for hybrid vehicles using V2V and V2I communications," Energy, Elsevier, vol. 225(C).
    2. Ganesh, Akhil Hannegudda & Xu, Bin, 2022. "A review of reinforcement learning based energy management systems for electrified powertrains: Progress, challenge, and potential solution," Renewable and Sustainable Energy Reviews, Elsevier, vol. 154(C).
    3. Zhang, Cetengfei & Zhou, Quan & Hua, Min & Xu, Hongming & Bassett, Mike & Zhang, Fanggang, 2023. "Cuboid equivalent consumption minimization strategy for energy management of multi-mode plug-in hybrid vehicles considering diverse time scale objectives," Applied Energy, Elsevier, vol. 351(C).
    4. Huang, Ruchen & He, Hongwen & Su, Qicong, 2024. "Towards a fossil-free urban transport system: An intelligent cross-type transferable energy management framework based on deep transfer reinforcement learning," Applied Energy, Elsevier, vol. 363(C).
    5. Zhang, Wei & Wang, Jixin & Xu, Zhenyu & Shen, Yuying & Gao, Guangzong, 2022. "A generalized energy management framework for hybrid construction vehicles via model-based reinforcement learning," Energy, Elsevier, vol. 260(C).
    6. He, Hongwen & Meng, Xiangfei & Wang, Yong & Khajepour, Amir & An, Xiaowen & Wang, Renguang & Sun, Fengchun, 2024. "Deep reinforcement learning based energy management strategies for electrified vehicles: Recent advances and perspectives," Renewable and Sustainable Energy Reviews, Elsevier, vol. 192(C).
    7. Lork, Clement & Li, Wen-Tai & Qin, Yan & Zhou, Yuren & Yuen, Chau & Tushar, Wayes & Saha, Tapan K., 2020. "An uncertainty-aware deep reinforcement learning framework for residential air conditioning energy management," Applied Energy, Elsevier, vol. 276(C).
    8. Wang, Hao & He, Hongwen & Bai, Yunfei & Yue, Hongwei, 2022. "Parameterized deep Q-network based energy management with balanced energy economy and battery life for hybrid electric vehicles," Applied Energy, Elsevier, vol. 320(C).
    9. Li, Jie & Fotouhi, Abbas & Liu, Yonggang & Zhang, Yuanjian & Chen, Zheng, 2024. "Review on eco-driving control for connected and automated vehicles," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PB).
    10. Hua, Min & Zhang, Cetengfei & Zhang, Fanggang & Li, Zhi & Yu, Xiaoli & Xu, Hongming & Zhou, Quan, 2023. "Energy management of multi-mode plug-in hybrid electric vehicle using multi-agent deep reinforcement learning," Applied Energy, Elsevier, vol. 348(C).
    11. Marc G. Bellemare & Salvatore Candido & Pablo Samuel Castro & Jun Gong & Marlos C. Machado & Subhodeep Moitra & Sameera S. Ponda & Ziyu Wang, 2020. "Autonomous navigation of stratospheric balloons using reinforcement learning," Nature, Nature, vol. 588(7836), pages 77-82, December.
    12. Song, Ke & Ding, Yuhang & Hu, Xiao & Xu, Hongjie & Wang, Yimin & Cao, Jing, 2021. "Degradation adaptive energy management strategy using fuel cell state-of-health for fuel economy improvement of hybrid electric vehicle," Applied Energy, Elsevier, vol. 285(C).
    13. Peter R. Wurman & Samuel Barrett & Kenta Kawamoto & James MacGlashan & Kaushik Subramanian & Thomas J. Walsh & Roberto Capobianco & Alisa Devlic & Franziska Eckert & Florian Fuchs & Leilani Gilpin & P, 2022. "Outracing champion Gran Turismo drivers with deep reinforcement learning," Nature, Nature, vol. 602(7896), pages 223-228, February.
    14. Tang, Xiaolin & Zhou, Haitao & Wang, Feng & Wang, Weida & Lin, Xianke, 2022. "Longevity-conscious energy management strategy of fuel cell hybrid electric Vehicle Based on deep reinforcement learning," Energy, Elsevier, vol. 238(PA).
    15. Wu, Jingda & Huang, Chao & He, Hongwen & Huang, Hailong, 2024. "Confidence-aware reinforcement learning for energy management of electrified vehicles," Renewable and Sustainable Energy Reviews, Elsevier, vol. 191(C).
    16. Lu, Dagang & Yi, Fengyan & Hu, Donghai & Li, Jianwei & Yang, Qingqing & Wang, Jing, 2023. "Online optimization of energy management strategy for FCV control parameters considering dual power source lifespan decay synergy," Applied Energy, Elsevier, vol. 348(C).
    17. Saiteja, Pemmareddy & Ashok, B., 2022. "Critical review on structural architecture, energy control strategies and development process towards optimal energy management in hybrid vehicles," Renewable and Sustainable Energy Reviews, Elsevier, vol. 157(C).
    18. Min, Haitao & Wu, Huiduo & Zhao, Honghui & Sun, Weiyi & Yu, Yuanbin, 2024. "Research on energy management strategy for fuel cell hybrid electric vehicles based on multi-scale information fusion," Applied Energy, Elsevier, vol. 368(C).
    19. Hu, Dong & Huang, Chao & Yin, Guodong & Li, Yangmin & Huang, Yue & Huang, Hailong & Wu, Jingda & Li, Wenfei & Xie, Hui, 2024. "A transfer-based reinforcement learning collaborative energy management strategy for extended-range electric buses with cabin temperature comfort consideration," Energy, Elsevier, vol. 290(C).
    20. Jia, Chunchun & He, Hongwen & Zhou, Jiaming & Li, Jianwei & Wei, Zhongbao & Li, Kunang, 2024. "Learning-based model predictive energy management for fuel cell hybrid electric bus with health-aware control," Applied Energy, Elsevier, vol. 355(C).
    21. Hu, Dong & Xie, Hui & Song, Kang & Zhang, Yuanyuan & Yan, Long, 2023. "An apprenticeship-reinforcement learning scheme based on expert demonstrations for energy management strategy of hybrid electric vehicles," Applied Energy, Elsevier, vol. 342(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hu, Dong & Huang, Chao & Yin, Guodong & Li, Yangmin & Huang, Yue & Huang, Hailong & Wu, Jingda & Li, Wenfei & Xie, Hui, 2024. "A transfer-based reinforcement learning collaborative energy management strategy for extended-range electric buses with cabin temperature comfort consideration," Energy, Elsevier, vol. 290(C).
    2. He, Hongwen & Meng, Xiangfei & Wang, Yong & Khajepour, Amir & An, Xiaowen & Wang, Renguang & Sun, Fengchun, 2024. "Deep reinforcement learning based energy management strategies for electrified vehicles: Recent advances and perspectives," Renewable and Sustainable Energy Reviews, Elsevier, vol. 192(C).
    3. Huang, Ruchen & He, Hongwen & Gao, Miaojue, 2023. "Training-efficient and cost-optimal energy management for fuel cell hybrid electric bus based on a novel distributed deep reinforcement learning framework," Applied Energy, Elsevier, vol. 346(C).
    4. Wu, Jingda & Huang, Chao & He, Hongwen & Huang, Hailong, 2024. "Confidence-aware reinforcement learning for energy management of electrified vehicles," Renewable and Sustainable Energy Reviews, Elsevier, vol. 191(C).
    5. Fan Wang & Yina Hong & Xiaohuan Zhao, 2025. "Research and Comparative Analysis of Energy Management Strategies for Hybrid Electric Vehicles: A Review," Energies, MDPI, vol. 18(11), pages 1-28, May.
    6. Li, Jianwei & Liu, Jie & Yang, Qingqing & Wang, Tianci & He, Hongwen & Wang, Hanxiao & Sun, Fengchun, 2025. "Reinforcement learning based energy management for fuel cell hybrid electric vehicles: A comprehensive review on decision process reformulation and strategy implementation," Renewable and Sustainable Energy Reviews, Elsevier, vol. 213(C).
    7. Zhang, Yahui & You, Xiongxiong & Song, Yunfeng & Zhao, Yahui & Wei, Zeyi & Jiao, Xiaohong, 2025. "Hierarchical eco-driving of connected hybrid electric vehicles: Integrating predictive cruise control and cost-to-go approximation-guided energy management," Energy, Elsevier, vol. 319(C).
    8. Huang, Ruchen & He, Hongwen & Su, Qicong & Härtl, Martin & Jaensch, Malte, 2025. "Type- and task-crossing energy management for fuel cell vehicles with longevity consideration: A heterogeneous deep transfer reinforcement learning framework," Applied Energy, Elsevier, vol. 377(PC).
    9. Zhang, Hao & Lei, Nuo & Chen, Boli & Li, Bingbing & Li, Rulong & Wang, Zhi, 2024. "Modeling and control system optimization for electrified vehicles: A data-driven approach," Energy, Elsevier, vol. 310(C).
    10. Yong Wang & Jingda Wu & Hongwen He & Zhongbao Wei & Fengchun Sun, 2025. "Data-driven energy management for electric vehicles using offline reinforcement learning," Nature Communications, Nature, vol. 16(1), pages 1-16, December.
    11. Zhao, Yinghua & Huang, Siqi & Wang, Xiaoyu & Shi, Jingwu & Yao, Shouwen, 2024. "Energy management with adaptive moving average filter and deep deterministic policy gradient reinforcement learning for fuel cell hybrid electric vehicles," Energy, Elsevier, vol. 312(C).
    12. Jinming Xu & Yuan Lin, 2024. "Energy Management for Hybrid Electric Vehicles Using Safe Hybrid-Action Reinforcement Learning," Mathematics, MDPI, vol. 12(5), pages 1-20, February.
    13. Hu, Dong & Xie, Hui & Song, Kang & Zhang, Yuanyuan & Yan, Long, 2023. "An apprenticeship-reinforcement learning scheme based on expert demonstrations for energy management strategy of hybrid electric vehicles," Applied Energy, Elsevier, vol. 342(C).
    14. Niu, Zegong & He, Hongwen, 2024. "A data-driven solution for intelligent power allocation of connected hybrid electric vehicles inspired by offline deep reinforcement learning in V2X scenario," Applied Energy, Elsevier, vol. 372(C).
    15. Cui, Wei & Cui, Naxin & Li, Tao & Du, Yi & Zhang, Chenghui, 2024. "Multi-objective hierarchical energy management for connected plug-in hybrid electric vehicle with cyber–physical interaction," Applied Energy, Elsevier, vol. 360(C).
    16. Tang, Tianfeng & Peng, Qianlong & Shi, Qing & Peng, Qingguo & Zhao, Jin & Chen, Chaoyi & Wang, Guangwei, 2024. "Energy management of fuel cell hybrid electric bus in mountainous regions: A deep reinforcement learning approach considering terrain characteristics," Energy, Elsevier, vol. 311(C).
    17. Gao, Sichen & Zong, Yuhua & Ju, Fei & Wang, Qun & Huo, Weiwei & Wang, Liangmo & Wang, Tao, 2024. "Scenario-oriented adaptive ECMS using speed prediction for fuel cell vehicles in real-world driving," Energy, Elsevier, vol. 304(C).
    18. Zhang, Dongfang & Sun, Wei & Zou, Yuan & Zhang, Xudong, 2025. "Energy management in HDHEV with dual APUs: Enhancing soft actor-critic using clustered experience replay and multi-dimensional priority sampling," Energy, Elsevier, vol. 319(C).
    19. Yan Tong & Issam Salhi & Qin Wang & Gang Lu & Shengyu Wu, 2025. "Bidirectional DC-DC Converter Topologies for Hybrid Energy Storage Systems in Electric Vehicles: A Comprehensive Review," Energies, MDPI, vol. 18(9), pages 1-29, May.
    20. Zhang, Baodi & Chang, Liang & Teng, Teng & Chen, Qifang & Li, Qiangwei & Cao, Yaoguang & Yang, Shichun & Zhang, Xin, 2024. "Multi-objective optimization with Q-learning for cruise and power allocation control parameters of connected fuel cell hybrid vehicles," Applied Energy, Elsevier, vol. 373(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:381:y:2025:i:c:s0306261924025224. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.