IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v286y2024ics0360544223028980.html

A multi-head attention mechanism aided hybrid network for identifying batteries’ state of charge

Author

Listed:
  • Li, Zongxiang
  • Li, Liwei
  • Chen, Jing
  • Wang, Dongqing

Abstract

A Convolution Neural Network and a Bidirectional Long Short-Term Memory network connected architecture with a Multi-Head Attention mechanism (CNN-BiLSTM-MHA) is studied for predicting state of charge (SOC) of lithium-ion batteries (LIBs). Firstly, an adaptive noise based complete ensemble empirical mode decomposition (AN-CEEMD) algorithm is adopted to catch the intrinsic features of measured battery signals by adding white noises. Secondly, a CNN-BiLSTM model with the MHA mechanism is developed to learn the mapping between processed input signals and battery SOC, it has three parts: 1) the CNN extracts features of the processed data, mine the relation among input signals, and promote estimation precision; 2) the BiLSTM has memory ability to catch battery dynamics, and the Swish activation function in the BiLSTM ensures unsaturated and reduces over-fitting due to its upper unbound and lower bound; 3) The multi-head attention mechanism uses several independent self-attention layers to associate input information and variables, extracts more associate information by adding weights; it reduces the overfitting risk of a single attention head, and improves the model generalization performance through joint learning of multiple heads. Finally, experiments and simulations are implemented under four operating conditions at five different temperatures, and the presented method is verified effective.

Suggested Citation

  • Li, Zongxiang & Li, Liwei & Chen, Jing & Wang, Dongqing, 2024. "A multi-head attention mechanism aided hybrid network for identifying batteries’ state of charge," Energy, Elsevier, vol. 286(C).
    • Handle: RePEc:eee:energy:v:286:y:2024:i:c:s0360544223028980
    DOI: 10.1016/j.energy.2023.129504
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544223028980
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2023.129504?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Zhao, Jian & Li, Xianguo & Shum, Chris & McPhee, John, 2023. "Control-oriented computational fuel cell dynamics modeling – Model order reduction vs. computational speed," Energy, Elsevier, vol. 266(C).
    2. Maosong Fan & Mengmeng Geng & Kai Yang & Mingjie Zhang & Hao Liu, 2023. "State of Health Estimation of Lithium-Ion Battery Based on Electrochemical Impedance Spectroscopy," Energies, MDPI, vol. 16(8), pages 1-14, April.
    3. Rezk, Hegazy & Olabi, A.G. & Ferahtia, Seydali & Sayed, Enas Taha, 2022. "Accurate parameter estimation methodology applied to model proton exchange membrane fuel cell," Energy, Elsevier, vol. 255(C).
    4. Wang, Xiaofei & Sun, Quan & Kou, Xiao & Ma, Wentao & Zhang, Hong & Liu, Rui, 2022. "Noise immune state of charge estimation of li-ion battery via the extreme learning machine with mixture generalized maximum correntropy criterion," Energy, Elsevier, vol. 239(PD).
    5. Atyabi, Seyed Ali & Afshari, Ebrahim & Shakarami, Negar, 2023. "Three-dimensional multiphase modeling of the performance of an open-cathode PEM fuel cell with additional cooling channels," Energy, Elsevier, vol. 263(PA).
    6. Momcilovic, Vladimir & Dimitrijevic, Branka & Stokic, Marko, 2023. "Supercapacitor electric bus modeling and simulation framework," Energy, Elsevier, vol. 282(C).
    7. Li, Jingmiao & Liu, Dehong, 2023. "Carbon price forecasting based on secondary decomposition and feature screening," Energy, Elsevier, vol. 278(PA).
    8. Ren, Xiaoqing & Liu, Shulin & Yu, Xiaodong & Dong, Xia, 2021. "A method for state-of-charge estimation of lithium-ion batteries based on PSO-LSTM," Energy, Elsevier, vol. 234(C).
    9. Takyi-Aninakwa, Paul & Wang, Shunli & Zhang, Hongying & Yang, Xiao & Fernandez, Carlos, 2023. "A hybrid probabilistic correction model for the state of charge estimation of lithium-ion batteries considering dynamic currents and temperatures," Energy, Elsevier, vol. 273(C).
    10. Gang Dou & Kaixuan Zhao & Mei Guo & Jun Mou, 2023. "Memristor-Based Lstm Network For Text Classification," FRACTALS (fractals), World Scientific Publishing Co. Pte. Ltd., vol. 31(06), pages 1-12.
    11. Oyewole, Isaiah & Chehade, Abdallah & Kim, Youngki, 2022. "A controllable deep transfer learning network with multiple domain adaptation for battery state-of-charge estimation," Applied Energy, Elsevier, vol. 312(C).
    12. Hu, Xiaosong & Jiang, Haifu & Feng, Fei & Liu, Bo, 2020. "An enhanced multi-state estimation hierarchy for advanced lithium-ion battery management," Applied Energy, Elsevier, vol. 257(C).
    13. Chaoran Li & Fei Xiao & Yaxiang Fan, 2019. "An Approach to State of Charge Estimation of Lithium-Ion Batteries Based on Recurrent Neural Networks with Gated Recurrent Unit," Energies, MDPI, vol. 12(9), pages 1-22, April.
    14. Xinwei Sun & Yang Zhang & Yongcheng Zhang & Licheng Wang & Kai Wang, 2023. "Summary of Health-State Estimation of Lithium-Ion Batteries Based on Electrochemical Impedance Spectroscopy," Energies, MDPI, vol. 16(15), pages 1-19, July.
    15. Jiang, Cong & Wang, Shunli & Wu, Bin & Fernandez, Carlos & Xiong, Xin & Coffie-Ken, James, 2021. "A state-of-charge estimation method of the power lithium-ion battery in complex conditions based on adaptive square root extended Kalman filter," Energy, Elsevier, vol. 219(C).
    16. Xing, Yinjiao & He, Wei & Pecht, Michael & Tsui, Kwok Leung, 2014. "State of charge estimation of lithium-ion batteries using the open-circuit voltage at various ambient temperatures," Applied Energy, Elsevier, vol. 113(C), pages 106-115.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Wang, Jun & Cao, Junxing, 2024. "Reservoir properties inversion using attention-based parallel hybrid network integrating feature selection and transfer learning," Energy, Elsevier, vol. 304(C).
    2. Chen, Haoyu & Huang, Hai & Zheng, Yong & Yang, Bing, 2024. "A load forecasting approach for integrated energy systems based on aggregation hybrid modal decomposition and combined model," Applied Energy, Elsevier, vol. 375(C).
    3. Yuan, Zhu & Deng, Zhongwei & He, Yvxin & Ning, Zhansheng & Liu, Jincheng, 2025. "Multi-step prediction of battery state of health based on self-supervised pre-training and transfer learning using the xPatch model," Energy, Elsevier, vol. 341(C).
    4. Zou, Yuanru & Shi, Haotian & Cao, Wen & Wang, Shunli & Nie, Shiliang & Zhang, Qin, 2025. "Enhanced group convolutional hybrid neural network for state of charge estimation of lithium-ion batteries consider temperature uncertainty," Energy, Elsevier, vol. 332(C).
    5. Fan, Yuqian & Yan, Chong & Wu, Xiaoying & Li, Yi & Dou, Wenwen & Gao, Guohong & Zhang, Pingchuan & Guan, Quanxue & Tan, Xiaojun, 2025. "Mechanical stress-based state-of-charge estimation for lithium-ion batteries via deep learning techniques," Energy, Elsevier, vol. 326(C).
    6. Zang, Haixiang & Li, Wenan & Cheng, Lilin & Liu, Jingxuan & Wei, Zhinong & Sun, Guoqiang, 2025. "Short-term multi-site solar irradiance prediction with dynamic-graph-convolution-based spatial-temporal correlation capturing," Renewable Energy, Elsevier, vol. 246(C).
    7. Wang, Zhuoer & Zhu, Xiaowen & Wang, Qingbo & Zhou, Jian & Li, Bijun & Shi, Baohan & Zhang, Chenming, 2025. "MapVC: Map-based deep learning for real-time current prediction in eco-driving electric vehicles," Applied Energy, Elsevier, vol. 396(C).
    8. Liu, Zhibin & Li, Lei & Ding, Xiaoyin & Wang, Xia & Liu, Zhiheng & Wang, Yawen & Hu, Changpeng, 2025. "Anomaly detection for real-world electric vehicle charging data using a convolutional autoencoder with multiscale convolution, attention mechanism, and BiLSTM," Energy, Elsevier, vol. 337(C).
    9. Wang, Jun & Qu, Ziyi & Zang, Haoyu & Bai, Kaiyu & Tian, Lijie & Zhang, Rui, 2026. "VMD-BiGRU-ATT architecture for aviation failure events prediction based on the grey wolf optimization algorithm," Reliability Engineering and System Safety, Elsevier, vol. 267(PB).
    10. Chen, Bingyang & Zeng, Xingjie & Liu, Chao & Xu, Yafei & Cao, Heling, 2025. "Health management of power batteries in low temperatures based on Adaptive Transfer Enformer framework," Reliability Engineering and System Safety, Elsevier, vol. 254(PA).
    11. Li, Penghua & Ye, Jiangtao & Hou, Jie & Deng, Zhongwei & Xiang, Sheng, 2025. "State of charge estimation for lithium-ion battery using a multi-feature Mamba network and UKF under mixed operating conditions," Energy, Elsevier, vol. 335(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Takyi-Aninakwa, Paul & Wang, Shunli & Zhang, Hongying & Yang, Xiao & Fernandez, Carlos, 2023. "A hybrid probabilistic correction model for the state of charge estimation of lithium-ion batteries considering dynamic currents and temperatures," Energy, Elsevier, vol. 273(C).
    2. Takyi-Aninakwa, Paul & Wang, Shunli & Zhang, Hongying & Yang, Xiaoyong & Fernandez, Carlos, 2022. "An optimized long short-term memory-weighted fading extended Kalman filtering model with wide temperature adaptation for the state of charge estimation of lithium-ion batteries," Applied Energy, Elsevier, vol. 326(C).
    3. Jiang, Han & Yin, Le & Xu, Zihan & Hu, Lizhou & Huang, Wei & Zhao, Yixin, 2025. "A novel hybrid framework for SOC estimation using PatchMixer-LSTM and adaptive UKF," Energy, Elsevier, vol. 335(C).
    4. Chen, Junxiong & Zhang, Yu & Wu, Ji & Cheng, Weisong & Zhu, Qiao, 2023. "SOC estimation for lithium-ion battery using the LSTM-RNN with extended input and constrained output," Energy, Elsevier, vol. 262(PA).
    5. Chen, Bingyang & Zeng, Xingjie & Liu, Chao & Xu, Yafei & Cao, Heling, 2025. "Health management of power batteries in low temperatures based on Adaptive Transfer Enformer framework," Reliability Engineering and System Safety, Elsevier, vol. 254(PA).
    6. Yang, Kuo & Tang, Yugui & Zhang, Shujing & Zhang, Zhen, 2022. "A deep learning approach to state of charge estimation of lithium-ion batteries based on dual-stage attention mechanism," Energy, Elsevier, vol. 244(PB).
    7. Takyi-Aninakwa, Paul & Wang, Shunli & Zhang, Hongying & Li, Huan & Xu, Wenhua & Fernandez, Carlos, 2022. "An optimized relevant long short-term memory-squared gain extended Kalman filter for the state of charge estimation of lithium-ion batteries," Energy, Elsevier, vol. 260(C).
    8. Zhang, Kai & Bai, Dongxin & Li, Yong & Song, Ke & Zheng, Bailin & Yang, Fuqian, 2024. "Robust state-of-charge estimator for lithium-ion batteries enabled by a physics-driven dual-stage attention mechanism," Applied Energy, Elsevier, vol. 359(C).
    9. Shehzar Shahzad Sheikh & Mahnoor Anjum & Muhammad Abdullah Khan & Syed Ali Hassan & Hassan Abdullah Khalid & Adel Gastli & Lazhar Ben-Brahim, 2020. "A Battery Health Monitoring Method Using Machine Learning: A Data-Driven Approach," Energies, MDPI, vol. 13(14), pages 1-16, July.
    10. Wang, Fu-Kwun & Kebede, Getnet Awoke & Lo, Shih-Che & Woldegiorgis, Bereket Haile, 2024. "An embedding layer-based quantum long short-term memory model with transfer learning for proton exchange membrane fuel stack remaining useful life prediction," Energy, Elsevier, vol. 308(C).
    11. Shahjalal, Mohammad & Roy, Probir Kumar & Shams, Tamanna & Fly, Ashley & Chowdhury, Jahedul Islam & Ahmed, Md. Rishad & Liu, Kailong, 2022. "A review on second-life of Li-ion batteries: prospects, challenges, and issues," Energy, Elsevier, vol. 241(C).
    12. Wan, Sicheng & Yang, Haojing & Lin, Jinwen & Li, Junhui & Wang, Yibo & Chen, Xinman, 2024. "Improved whale optimization algorithm towards precise state-of-charge estimation of lithium-ion batteries via optimizing LSTM," Energy, Elsevier, vol. 310(C).
    13. Zhang, Hao & Gao, Jingyi & Kang, Le & Zhang, Yi & Wang, Licheng & Wang, Kai, 2023. "State of health estimation of lithium-ion batteries based on modified flower pollination algorithm-temporal convolutional network," Energy, Elsevier, vol. 283(C).
    14. Liu, Zheng & Zhao, Zhenhua & Qiu, Yuan & Jing, Benqin & Yang, Chunshan & Wu, Huifeng, 2023. "Enhanced state of charge estimation for Li-ion batteries through adaptive maximum correntropy Kalman filter with open circuit voltage correction," Energy, Elsevier, vol. 283(C).
    15. Hu, Jingwei & Li, Xiaodong & Fang, Zheng & Cheng, Jun & Yi, Longqiang & Zhang, Zhihong, 2025. "Estimate state of charge in lithium-ion batteries with unknown data," Applied Energy, Elsevier, vol. 389(C).
    16. Jin, Zhaorui & Fu, Shiyi & Fan, Hongtao & Tao, Yulin & Dong, Yachao & Wang, Yu & Sun, Yaojie, 2025. "Edge-cloud collaborative method for state of charge estimation of lithium-ion batteries by combining Kalman filter and deep learning," Energy, Elsevier, vol. 332(C).
    17. Chen, Xiaohui & Yang, Haixu & Pan, Chenyang & Jia, Zirun & Wang, Zhenpo, 2025. "A vehicle-cloud collaborative framework for state of health estimation of lithium-ion batteries via multi-feature fusion and hybrid data-driven–empirical modeling," Energy, Elsevier, vol. 340(C).
    18. Jiang, Bo & Tao, Siyi & Wang, Xueyuan & Zhu, Jiangong & Wei, Xuezhe & Dai, Haifeng, 2023. "Mechanics-based state of charge estimation for lithium-ion pouch battery using deep learning technique," Energy, Elsevier, vol. 278(PA).
    19. Wang, Yonggang & Yu, Yadong & Ma, Yuanchu & Shi, Jie, 2025. "Lithium-ion battery health state estimation based on improved snow ablation optimization algorithm-deep hybrid kernel extreme learning machine," Energy, Elsevier, vol. 323(C).
    20. Chen, Guisheng & Zhou, Hengyu & Yang, Jie & Xu, Yangsong & Xiao, Renxin & Wang, Yusong, 2026. "Optimization of open-circuit voltage curve acquisition and state estimation for aging lithium-ion batteries," Energy, Elsevier, vol. 342(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:286:y:2024:i:c:s0360544223028980. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.