IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v312y2024ics0360544224031712.html
   My bibliography  Save this article

Energy management with adaptive moving average filter and deep deterministic policy gradient reinforcement learning for fuel cell hybrid electric vehicles

Author

Listed:
  • Zhao, Yinghua
  • Huang, Siqi
  • Wang, Xiaoyu
  • Shi, Jingwu
  • Yao, Shouwen

Abstract

Fuel cell hybrid electric vehicles (FCHEV) with battery (BAT) and supercapacitor (SC) advance in flexible configuration and high energy efficiency. However, the complex coupling relationship among various power sources poses a severe challenge to the design of the energy management system (EMS), including multi-degrees of freedom power allocation, fuel economy, and power sources lifespan of the FCHEV. This paper proposes an EMS based on a dual-layer power distribution structure. In the upper layer, adaptive moving average filter (AMAF) is designed to separate different frequency power, where the energy supply of the SC is managed to attenuate fluctuating power and simplifies the optimization problem and reduces computational costs. The lower layer is constructed by the deep deterministic policy gradient (DDPG) algorithm, where fuel cell system (FCS) hydrogen consumption and degradation rewards are designed to simultaneously enhance fuel efficiency and degradation performance by regulating the FCS real-time power variation. The proposed strategy has been evaluated regarding FCHEV fuel economy and FCS durability under combined driving cycle simulation, which shows AMAF + DDPG strategy reduces fuel consumption by 7.24 % and 1.3 %, also the degradation reduces by 0.04 % and 0.02 % compared with different EMS. Simulation results demonstrate that AMAF + DDPG optimizes the output characteristics of power sources.

Suggested Citation

  • Zhao, Yinghua & Huang, Siqi & Wang, Xiaoyu & Shi, Jingwu & Yao, Shouwen, 2024. "Energy management with adaptive moving average filter and deep deterministic policy gradient reinforcement learning for fuel cell hybrid electric vehicles," Energy, Elsevier, vol. 312(C).
    • Handle: RePEc:eee:energy:v:312:y:2024:i:c:s0360544224031712
    DOI: 10.1016/j.energy.2024.133395
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544224031712
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2024.133395?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Jia, Chunchun & Zhou, Jiaming & He, Hongwen & Li, Jianwei & Wei, Zhongbao & Li, Kunang, 2024. "Health-conscious deep reinforcement learning energy management for fuel cell buses integrating environmental and look-ahead road information," Energy, Elsevier, vol. 290(C).
    2. Peng, Jiankun & Shen, Yang & Wu, ChangCheng & Wang, Chunhai & Yi, Fengyan & Ma, Chunye, 2023. "Research on energy-saving driving control of hydrogen fuel bus based on deep reinforcement learning in freeway ramp weaving area," Energy, Elsevier, vol. 285(C).
    3. Tang, Xiaolin & Zhou, Haitao & Wang, Feng & Wang, Weida & Lin, Xianke, 2022. "Longevity-conscious energy management strategy of fuel cell hybrid electric Vehicle Based on deep reinforcement learning," Energy, Elsevier, vol. 238(PA).
    4. Wu, Yuankai & Tan, Huachun & Peng, Jiankun & Zhang, Hailong & He, Hongwen, 2019. "Deep reinforcement learning of energy management with continuous control strategy and traffic information for a series-parallel plug-in hybrid electric bus," Applied Energy, Elsevier, vol. 247(C), pages 454-466.
    5. İnci, Mustafa & Büyük, Mehmet & Demir, Mehmet Hakan & İlbey, Göktürk, 2021. "A review and research on fuel cell electric vehicles: Topologies, power electronic converters, energy management methods, technical challenges, marketing and future aspects," Renewable and Sustainable Energy Reviews, Elsevier, vol. 137(C).
    6. Shabbir, Wassif & Evangelou, Simos A., 2014. "Real-time control strategy to maximize hybrid electric vehicle powertrain efficiency," Applied Energy, Elsevier, vol. 135(C), pages 512-522.
    7. Jiang, Hongliang & Xu, Liangfei & Li, Jianqiu & Hu, Zunyan & Ouyang, Minggao, 2019. "Energy management and component sizing for a fuel cell/battery/supercapacitor hybrid powertrain based on two-dimensional optimization algorithms," Energy, Elsevier, vol. 177(C), pages 386-396.
    8. Liu, Teng & Wang, Bo & Yang, Chenglang, 2018. "Online Markov Chain-based energy management for a hybrid tracked vehicle with speedy Q-learning," Energy, Elsevier, vol. 160(C), pages 544-555.
    9. Lu, Dagang & Yi, Fengyan & Hu, Donghai & Li, Jianwei & Yang, Qingqing & Wang, Jing, 2023. "Online optimization of energy management strategy for FCV control parameters considering dual power source lifespan decay synergy," Applied Energy, Elsevier, vol. 348(C).
    10. Jia, Chunchun & He, Hongwen & Zhou, Jiaming & Li, Jianwei & Wei, Zhongbao & Li, Kunang, 2024. "Learning-based model predictive energy management for fuel cell hybrid electric bus with health-aware control," Applied Energy, Elsevier, vol. 355(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mounika, Kandi & Bhattacharjee, Ankur, 2025. "Design and experimental validation for performance analysis of non-isolated power converter topologies in fuel cell integrated dynamic load based local energy systems," Energy, Elsevier, vol. 322(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Li, Jianwei & Liu, Jie & Yang, Qingqing & Wang, Tianci & He, Hongwen & Wang, Hanxiao & Sun, Fengchun, 2025. "Reinforcement learning based energy management for fuel cell hybrid electric vehicles: A comprehensive review on decision process reformulation and strategy implementation," Renewable and Sustainable Energy Reviews, Elsevier, vol. 213(C).
    2. Zhang, Yahui & You, Xiongxiong & Song, Yunfeng & Zhao, Yahui & Wei, Zeyi & Jiao, Xiaohong, 2025. "Hierarchical eco-driving of connected hybrid electric vehicles: Integrating predictive cruise control and cost-to-go approximation-guided energy management," Energy, Elsevier, vol. 319(C).
    3. Yan Tong & Issam Salhi & Qin Wang & Gang Lu & Shengyu Wu, 2025. "Bidirectional DC-DC Converter Topologies for Hybrid Energy Storage Systems in Electric Vehicles: A Comprehensive Review," Energies, MDPI, vol. 18(9), pages 1-29, May.
    4. Yang Shen & Jiaming Zhou & Jinming Zhang & Fengyan Yi & Guofeng Wang & Chaofeng Pan & Wei Guo & Xing Shu, 2023. "Research on Energy Management of Hydrogen Fuel Cell Bus Based on Deep Reinforcement Learning Considering Velocity Control," Sustainability, MDPI, vol. 15(16), pages 1-19, August.
    5. Wang, Zhiguo & Wei, Hongqian & Xi, Yecheng & Xiao, Gongwei, 2024. "Data-driven energy utilization for plug-in hybrid electric bus with driving patten application and battery health considerations," Energy, Elsevier, vol. 310(C).
    6. Wu, Xiaohua & Ma, Pengfei & Zhou, Lingxue & Fan, Zhanfeng & Zhu, Qingbo & Yin, Xiaofeng & Ouyang, Quan, 2025. "Collaborative optimization of velocity planning and energy management for fuel cell hybrid buses at multiple intersections and bus stations," Energy, Elsevier, vol. 326(C).
    7. Yanwei Liu & Mingda Wang & Jialuo Tan & Jie Ye & Jiansheng Liang, 2024. "Real-Time Energy Management Strategy for Fuel Cell Vehicles Based on DP and Rule Extraction," Energies, MDPI, vol. 17(14), pages 1-20, July.
    8. Hu, Dong & Huang, Chao & Wu, Jingda & Wei, Henglai & Pi, Dawei, 2025. "Enhancing data-driven energy management strategy via digital expert guidance for electrified vehicles," Applied Energy, Elsevier, vol. 381(C).
    9. Ziye Wang & Ren He & Donghai Hu & Dagang Lu, 2025. "Energy Management Strategy for Fuel Cell Vehicles Based on Deep Transfer Reinforcement Learning," Energies, MDPI, vol. 18(9), pages 1-18, April.
    10. Matteo Acquarone & Claudio Maino & Daniela Misul & Ezio Spessa & Antonio Mastropietro & Luca Sorrentino & Enrico Busto, 2023. "Influence of the Reward Function on the Selection of Reinforcement Learning Agents for Hybrid Electric Vehicles Real-Time Control," Energies, MDPI, vol. 16(6), pages 1-22, March.
    11. Yang, Ningkang & Han, Lijin & Xiang, Changle & Liu, Hui & Li, Xunmin, 2021. "An indirect reinforcement learning based real-time energy management strategy via high-order Markov Chain model for a hybrid electric vehicle," Energy, Elsevier, vol. 236(C).
    12. Du, Guodong & Zou, Yuan & Zhang, Xudong & Kong, Zehui & Wu, Jinlong & He, Dingbo, 2019. "Intelligent energy management for hybrid electric tracked vehicles using online reinforcement learning," Applied Energy, Elsevier, vol. 251(C), pages 1-1.
    13. Tang, Tianfeng & Peng, Qianlong & Shi, Qing & Peng, Qingguo & Zhao, Jin & Chen, Chaoyi & Wang, Guangwei, 2024. "Energy management of fuel cell hybrid electric bus in mountainous regions: A deep reinforcement learning approach considering terrain characteristics," Energy, Elsevier, vol. 311(C).
    14. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    15. Song, Dafeng & Wu, Qingtao & Zeng, Xiaohua & Zhang, Xuanming & Qian, Qifeng & Yang, DongPo, 2024. "Feedback-linearization decoupling based coordinated control of air supply and thermal management for vehicular fuel cell system," Energy, Elsevier, vol. 305(C).
    16. Zhang, Dongfang & Sun, Wei & Zou, Yuan & Zhang, Xudong, 2025. "Energy management in HDHEV with dual APUs: Enhancing soft actor-critic using clustered experience replay and multi-dimensional priority sampling," Energy, Elsevier, vol. 319(C).
    17. Li, Jianwei & Wang, Tianci & Yang, Qingqing & Tian, Zhonghao & Lv, Hong & Wang, Xuechao & Shen, Jun, 2025. "A safe region method to quantitatively evaluate the safety of fuel cell operating states," Applied Energy, Elsevier, vol. 377(PA).
    18. Jia, Chunchun & He, Hongwen & Zhou, Jiaming & Li, Jianwei & Wei, Zhongbao & Li, Kunang, 2023. "A novel health-aware deep reinforcement learning energy management for fuel cell bus incorporating offline high-quality experience," Energy, Elsevier, vol. 282(C).
    19. Zhang, Dongfang & Sun, Wei & Zou, Yuan & Zhang, Xudong & Zhang, Yiwei, 2024. "An improved soft actor-critic-based energy management strategy of heavy-duty hybrid electric vehicles with dual-engine system," Energy, Elsevier, vol. 308(C).
    20. Xun, Qian & Murgovski, Nikolce & Liu, Yujing, 2022. "Chance-constrained robust co-design optimization for fuel cell hybrid electric trucks," Applied Energy, Elsevier, vol. 320(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:312:y:2024:i:c:s0360544224031712. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.