IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v293y2021ics0306261921004499.html
   My bibliography  Save this article

Cloud-based health-conscious energy management of hybrid battery systems in electric vehicles with deep reinforcement learning

Author

Listed:
  • Li, Weihan
  • Cui, Han
  • Nemeth, Thomas
  • Jansen, Jonathan
  • Ünlübayir, Cem
  • Wei, Zhongbao
  • Feng, Xuning
  • Han, Xuebing
  • Ouyang, Minggao
  • Dai, Haifeng
  • Wei, Xuezhe
  • Sauer, Dirk Uwe

Abstract

In order to fulfill the energy and power demand of battery electric vehicles, a hybrid battery system with a high-energy and a high-power battery pack can be implemented as the energy source. This paper explores a cloud-based multi-objective energy management strategy for the hybrid architecture with a deep deterministic policy gradient, which increases the electrical and thermal safety, and meanwhile minimizes the system’s energy loss and aging cost. In order to simulate the electro-thermal dynamics and aging behaviors of the batteries, models are built for both high-energy and high-power cells based on the characterization and aging tests. A cloud-based training approach is proposed for energy management with real-world vehicle data collected from various road conditions. Results show the improvement of electrical and thermal safety, as well as the reduction of energy loss and aging cost of the whole system with the proposed strategy based on the collected real-world driving data. Furthermore, processor-in-the-loop tests verify that the proposed strategy can achieve a much higher convergence rate and a better performance in terms of the minimization of both energy loss and aging cost compared with state-of-the-art learning-based strategies.

Suggested Citation

  • Li, Weihan & Cui, Han & Nemeth, Thomas & Jansen, Jonathan & Ünlübayir, Cem & Wei, Zhongbao & Feng, Xuning & Han, Xuebing & Ouyang, Minggao & Dai, Haifeng & Wei, Xuezhe & Sauer, Dirk Uwe, 2021. "Cloud-based health-conscious energy management of hybrid battery systems in electric vehicles with deep reinforcement learning," Applied Energy, Elsevier, vol. 293(C).
    • Handle: RePEc:eee:appene:v:293:y:2021:i:c:s0306261921004499
    DOI: 10.1016/j.apenergy.2021.116977
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261921004499
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2021.116977?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Li, Weihan & Cao, Decheng & Jöst, Dominik & Ringbeck, Florian & Kuipers, Matthias & Frie, Fabian & Sauer, Dirk Uwe, 2020. "Parameter sensitivity analysis of electrochemical model-based battery management systems for lithium-ion batteries," Applied Energy, Elsevier, vol. 269(C).
    2. Yuping Zeng & Yang Cai & Guiyue Kou & Wei Gao & Datong Qin, 2018. "Energy Management for Plug-In Hybrid Electric Vehicle Based on Adaptive Simplified-ECMS," Sustainability, MDPI, vol. 10(6), pages 1-24, June.
    3. Liu, Sijia & Winter, Michaela & Lewerenz, Meinert & Becker, Jan & Sauer, Dirk Uwe & Ma, Zeyu & Jiang, Jiuchun, 2019. "Analysis of cyclic aging performance of commercial Li4Ti5O12-based batteries at room temperature," Energy, Elsevier, vol. 173(C), pages 1041-1053.
    4. Shuxian Li & Minghui Hu & Changchao Gong & Sen Zhan & Datong Qin, 2018. "Energy Management Strategy for Hybrid Electric Vehicle Based on Driving Condition Identification Using KGA-Means," Energies, MDPI, vol. 11(6), pages 1-16, June.
    5. Yang, Chao & Du, Siyu & Li, Liang & You, Sixong & Yang, Yiyong & Zhao, Yue, 2017. "Adaptive real-time optimal energy management strategy based on equivalent factors optimization for plug-in hybrid electric vehicle," Applied Energy, Elsevier, vol. 203(C), pages 883-896.
    6. Wu, Jingda & He, Hongwen & Peng, Jiankun & Li, Yuecheng & Li, Zhanjiang, 2018. "Continuous reinforcement learning of energy management with deep Q network for a power split hybrid electric bus," Applied Energy, Elsevier, vol. 222(C), pages 799-811.
    7. Xiong, Rui & Cao, Jiayi & Yu, Quanqing, 2018. "Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle," Applied Energy, Elsevier, vol. 211(C), pages 538-548.
    8. Farouk Odeim & Jürgen Roes & Angelika Heinzel, 2015. "Power Management Optimization of an Experimental Fuel Cell/Battery/Supercapacitor Hybrid System," Energies, MDPI, vol. 8(7), pages 1-26, June.
    9. Veneri, Ottorino & Capasso, Clemente & Patalano, Stanislao, 2018. "Experimental investigation into the effectiveness of a super-capacitor based hybrid energy storage system for urban commercial vehicles," Applied Energy, Elsevier, vol. 227(C), pages 312-323.
    10. Shen, Peihong & Zhao, Zhiguo & Zhan, Xiaowen & Li, Jingwei & Guo, Qiuyi, 2018. "Optimal energy management strategy for a plug-in hybrid electric commercial vehicle based on velocity prediction," Energy, Elsevier, vol. 155(C), pages 838-852.
    11. Wang, Shuoqi & Guo, Dongxu & Han, Xuebing & Lu, Languang & Sun, Kai & Li, Weihan & Sauer, Dirk Uwe & Ouyang, Minggao, 2020. "Impact of battery degradation models on energy management of a grid-connected DC microgrid," Energy, Elsevier, vol. 207(C).
    12. Han, Xuefeng & He, Hongwen & Wu, Jingda & Peng, Jiankun & Li, Yuecheng, 2019. "Energy management based on reinforcement learning with double deep Q-learning for a hybrid electric tracked vehicle," Applied Energy, Elsevier, vol. 254(C).
    13. Mahmoudzadeh Andwari, Amin & Pesiridis, Apostolos & Rajoo, Srithar & Martinez-Botas, Ricardo & Esfahanian, Vahid, 2017. "A review of Battery Electric Vehicle technology and readiness levels," Renewable and Sustainable Energy Reviews, Elsevier, vol. 78(C), pages 414-430.
    14. Li, Yuecheng & He, Hongwen & Khajepour, Amir & Wang, Hong & Peng, Jiankun, 2019. "Energy management for a power-split hybrid electric bus via deep reinforcement learning with terrain information," Applied Energy, Elsevier, vol. 255(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhou, Quan & Li, Yanfei & Zhao, Dezong & Li, Ji & Williams, Huw & Xu, Hongming & Yan, Fuwu, 2022. "Transferable representation modelling for real-time energy management of the plug-in hybrid vehicle based on k-fold fuzzy learning and Gaussian process regression," Applied Energy, Elsevier, vol. 305(C).
    2. Naseri, F. & Gil, S. & Barbu, C. & Cetkin, E. & Yarimca, G. & Jensen, A.C. & Larsen, P.G. & Gomes, C., 2023. "Digital twin of electric vehicle battery systems: Comprehensive review of the use cases, requirements, and platforms," Renewable and Sustainable Energy Reviews, Elsevier, vol. 179(C).
    3. Yu, Xiao & Lin, Cheng & Xie, Peng & Liang, Sheng, 2022. "A novel real-time energy management strategy based on Monte Carlo Tree Search for coupled powertrain platform via vehicle-to-cloud connectivity," Energy, Elsevier, vol. 256(C).
    4. Yu, Xiao & Lin, Cheng & Tian, Yu & Zhao, Mingjie & Liu, Huimin & Xie, Peng & Zhang, JunZhi, 2023. "Real-time and hierarchical energy management-control framework for electric vehicles with dual-motor powertrain system," Energy, Elsevier, vol. 272(C).
    5. Rajput, Daizy & Herreros, Jose M. & Innocente, Mauro S. & Bryans, Jeremy & Schaub, Joschka & Dizqah, Arash M., 2022. "Impact of the number of planetary gears on the energy efficiency of electrified powertrains," Applied Energy, Elsevier, vol. 323(C).
    6. Li, Weihan & Fan, Yue & Ringbeck, Florian & Jöst, Dominik & Sauer, Dirk Uwe, 2022. "Unlocking electrochemical model-based online power prediction for lithium-ion batteries via Gaussian process regression," Applied Energy, Elsevier, vol. 306(PB).
    7. Chen, Jiaxin & Shu, Hong & Tang, Xiaolin & Liu, Teng & Wang, Weida, 2022. "Deep reinforcement learning-based multi-objective control of hybrid power system combined with road recognition under time-varying environment," Energy, Elsevier, vol. 239(PC).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wang, Yue & Li, Keqiang & Zeng, Xiaohua & Gao, Bolin & Hong, Jichao, 2023. "Investigation of novel intelligent energy management strategies for connected HEB considering global planning of fixed-route information," Energy, Elsevier, vol. 263(PB).
    2. Yang, Ningkang & Han, Lijin & Xiang, Changle & Liu, Hui & Li, Xunmin, 2021. "An indirect reinforcement learning based real-time energy management strategy via high-order Markov Chain model for a hybrid electric vehicle," Energy, Elsevier, vol. 236(C).
    3. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    4. Wu, Yuankai & Tan, Huachun & Peng, Jiankun & Zhang, Hailong & He, Hongwen, 2019. "Deep reinforcement learning of energy management with continuous control strategy and traffic information for a series-parallel plug-in hybrid electric bus," Applied Energy, Elsevier, vol. 247(C), pages 454-466.
    5. Marouane Adnane & Ahmed Khoumsi & João Pedro F. Trovão, 2023. "Efficient Management of Energy Consumption of Electric Vehicles Using Machine Learning—A Systematic and Comprehensive Survey," Energies, MDPI, vol. 16(13), pages 1-39, June.
    6. Vázquez-Canteli, José R. & Nagy, Zoltán, 2019. "Reinforcement learning for demand response: A review of algorithms and modeling techniques," Applied Energy, Elsevier, vol. 235(C), pages 1072-1089.
    7. Wu, Peng & Partridge, Julius & Bucknall, Richard, 2020. "Cost-effective reinforcement learning energy management for plug-in hybrid fuel cell and battery ships," Applied Energy, Elsevier, vol. 275(C).
    8. Fengqi Zhang & Lihua Wang & Serdar Coskun & Hui Pang & Yahui Cui & Junqiang Xi, 2020. "Energy Management Strategies for Hybrid Electric Vehicles: Review, Classification, Comparison, and Outlook," Energies, MDPI, vol. 13(13), pages 1-35, June.
    9. Shi, Wenzhuo & Huangfu, Yigeng & Xu, Liangcai & Pang, Shengzhao, 2022. "Online energy management strategy considering fuel cell fault for multi-stack fuel cell hybrid vehicle based on multi-agent reinforcement learning," Applied Energy, Elsevier, vol. 328(C).
    10. Huang, Ruchen & He, Hongwen & Zhao, Xuyang & Wang, Yunlong & Li, Menglin, 2022. "Battery health-aware and naturalistic data-driven energy management for hybrid electric bus based on TD3 deep reinforcement learning algorithm," Applied Energy, Elsevier, vol. 321(C).
    11. Chen, Zheng & Gu, Hongji & Shen, Shiquan & Shen, Jiangwei, 2022. "Energy management strategy for power-split plug-in hybrid electric vehicle based on MPC and double Q-learning," Energy, Elsevier, vol. 245(C).
    12. Dong, Peng & Zhao, Junwei & Liu, Xuewu & Wu, Jian & Xu, Xiangyang & Liu, Yanfang & Wang, Shuhan & Guo, Wei, 2022. "Practical application of energy management strategy for hybrid electric vehicles based on intelligent and connected technologies: Development stages, challenges, and future trends," Renewable and Sustainable Energy Reviews, Elsevier, vol. 170(C).
    13. Xiao, Boyi & Yang, Weiwei & Wu, Jiamin & Walker, Paul D. & Zhang, Nong, 2022. "Energy management strategy via maximum entropy reinforcement learning for an extended range logistics vehicle," Energy, Elsevier, vol. 253(C).
    14. Geng, Wenran & Lou, Diming & Wang, Chen & Zhang, Tong, 2020. "A cascaded energy management optimization method of multimode power-split hybrid electric vehicles," Energy, Elsevier, vol. 199(C).
    15. Liu, Teng & Tan, Wenhao & Tang, Xiaolin & Zhang, Jinwei & Xing, Yang & Cao, Dongpu, 2021. "Driving conditions-driven energy management strategies for hybrid electric vehicles: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 151(C).
    16. Wang, Siyang & Lin, Xianke, 2020. "Eco-driving control of connected and automated hybrid vehicles in mixed driving scenarios," Applied Energy, Elsevier, vol. 271(C).
    17. Xu, Bin & Rathod, Dhruvang & Zhang, Darui & Yebi, Adamu & Zhang, Xueyu & Li, Xiaoya & Filipi, Zoran, 2020. "Parametric study on reinforcement learning optimized energy management strategy for a hybrid electric vehicle," Applied Energy, Elsevier, vol. 259(C).
    18. Tran, Dai-Duong & Vafaeipour, Majid & El Baghdadi, Mohamed & Barrero, Ricardo & Van Mierlo, Joeri & Hegazy, Omar, 2020. "Thorough state-of-the-art analysis of electric and hybrid vehicle powertrains: Topologies and integrated energy management strategies," Renewable and Sustainable Energy Reviews, Elsevier, vol. 119(C).
    19. Diming Lou & Yinghua Zhao & Liang Fang & Yuanzhi Tang & Caihua Zhuang, 2022. "Encoder–Decoder-Based Velocity Prediction Modelling for Passenger Vehicles Coupled with Driving Pattern Recognition," Sustainability, MDPI, vol. 14(17), pages 1-21, August.
    20. Zhang, Wei & Wang, Jixin & Xu, Zhenyu & Shen, Yuying & Gao, Guangzong, 2022. "A generalized energy management framework for hybrid construction vehicles via model-based reinforcement learning," Energy, Elsevier, vol. 260(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:293:y:2021:i:c:s0306261921004499. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.