IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v17y2023i1p79-d1305634.html
   My bibliography  Save this article

Deep Reinforcement Learning-Based Battery Management Algorithm for Retired Electric Vehicle Batteries with a Heterogeneous State of Health in BESSs

Author

Listed:
  • Nhat Quang Doan

    (Department of Electrical, Electronic and Computer Engineering, University of Ulsan, Ulsan 44610, Republic of Korea)

  • Syed Maaz Shahid

    (Department of Electrical, Electronic and Computer Engineering, University of Ulsan, Ulsan 44610, Republic of Korea)

  • Sung-Jin Choi

    (Department of Electrical, Electronic and Computer Engineering, University of Ulsan, Ulsan 44610, Republic of Korea)

  • Sungoh Kwon

    (Department of Electrical, Electronic and Computer Engineering, University of Ulsan, Ulsan 44610, Republic of Korea)

Abstract

In this paper, we propose a battery management algorithm to optimize the lifetimes of retired lithium batteries with heterogeneous states of health in a battery energy storage system under dynamic power demand. A battery energy storage system allows for the use of retired lithium batteries for applications such as backup power in homes, data centers, etc. In a battery energy storage system, a battery pack consists of several retired batteries connected in parallel or in series to fulfill the required power demand. Owing to the retired batteries’ different capacity levels, i.e., states of health, a scheduling strategy is required to turn battery cells inside the battery pack on and off such that the secondary lifetimes of the retired batteries are extended. To establish the optimal scheduling policy, it is necessary to determine the correct states of each battery cell, including the state of charge and the state of health. To that end, the proposed algorithm first estimates the state of charge and state of health for all cells based on data measured using an extended Kalman filter. Then, a deep reinforcement learning scheduling algorithm is implemented to connect/disconnect the battery cells to/from the battery pack based on their states. Via simulation, we show that the proposed algorithm estimates the state of charge and state of health of each battery cell with low error and extends the lifetime of battery packs by 20.6%, compared to methods proposed in previous works.

Suggested Citation

  • Nhat Quang Doan & Syed Maaz Shahid & Sung-Jin Choi & Sungoh Kwon, 2023. "Deep Reinforcement Learning-Based Battery Management Algorithm for Retired Electric Vehicle Batteries with a Heterogeneous State of Health in BESSs," Energies, MDPI, vol. 17(1), pages 1-19, December.
    • Handle: RePEc:gam:jeners:v:17:y:2023:i:1:p:79-:d:1305634
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/17/1/79/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/17/1/79/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Song, Yuchen & Liu, Datong & Liao, Haitao & Peng, Yu, 2020. "A hybrid statistical data-driven method for on-line joint state estimation of lithium-ion batteries," Applied Energy, Elsevier, vol. 261(C).
    2. Ng, Kong Soon & Moo, Chin-Sien & Chen, Yi-Ping & Hsieh, Yao-Ching, 2009. "Enhanced coulomb counting method for estimating state-of-charge and state-of-health of lithium-ion batteries," Applied Energy, Elsevier, vol. 86(9), pages 1506-1511, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Panagiotis Eleftheriadis & Spyridon Giazitzis & Sonia Leva & Emanuele Ogliari, 2023. "Data-Driven Methods for the State of Charge Estimation of Lithium-Ion Batteries: An Overview," Forecasting, MDPI, vol. 5(3), pages 1-24, September.
    2. Chen, Junxiong & Zhang, Yu & Wu, Ji & Cheng, Weisong & Zhu, Qiao, 2023. "SOC estimation for lithium-ion battery using the LSTM-RNN with extended input and constrained output," Energy, Elsevier, vol. 262(PA).
    3. Sui, Xin & He, Shan & Vilsen, Søren B. & Meng, Jinhao & Teodorescu, Remus & Stroe, Daniel-Ioan, 2021. "A review of non-probabilistic machine learning-based state of health estimation techniques for Lithium-ion battery," Applied Energy, Elsevier, vol. 300(C).
    4. Semeraro, Concetta & Caggiano, Mariateresa & Olabi, Abdul-Ghani & Dassisti, Michele, 2022. "Battery monitoring and prognostics optimization techniques: Challenges and opportunities," Energy, Elsevier, vol. 255(C).
    5. Wang, Yujie & Tian, Jiaqiang & Sun, Zhendong & Wang, Li & Xu, Ruilong & Li, Mince & Chen, Zonghai, 2020. "A comprehensive review of battery modeling and state estimation approaches for advanced battery management systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 131(C).
    6. Wei, Jingwen & Chen, Chunlin, 2021. "A multi-timescale framework for state monitoring and lifetime prognosis of lithium-ion batteries," Energy, Elsevier, vol. 229(C).
    7. Shehzar Shahzad Sheikh & Mahnoor Anjum & Muhammad Abdullah Khan & Syed Ali Hassan & Hassan Abdullah Khalid & Adel Gastli & Lazhar Ben-Brahim, 2020. "A Battery Health Monitoring Method Using Machine Learning: A Data-Driven Approach," Energies, MDPI, vol. 13(14), pages 1-16, July.
    8. Yang, Duo & Wang, Yujie & Pan, Rui & Chen, Ruiyang & Chen, Zonghai, 2018. "State-of-health estimation for the lithium-ion battery based on support vector regression," Applied Energy, Elsevier, vol. 227(C), pages 273-283.
    9. Wenxian Duan & Chuanxue Song & Silun Peng & Feng Xiao & Yulong Shao & Shixin Song, 2020. "An Improved Gated Recurrent Unit Network Model for State-of-Charge Estimation of Lithium-Ion Battery," Energies, MDPI, vol. 13(23), pages 1-19, December.
    10. Guoqing Jin & Lan Li & Yidan Xu & Minghui Hu & Chunyun Fu & Datong Qin, 2020. "Comparison of SOC Estimation between the Integer-Order Model and Fractional-Order Model Under Different Operating Conditions," Energies, MDPI, vol. 13(7), pages 1-17, April.
    11. Hu, Lin & Hu, Xiaosong & Che, Yunhong & Feng, Fei & Lin, Xianke & Zhang, Zhiyong, 2020. "Reliable state of charge estimation of battery packs using fuzzy adaptive federated filtering," Applied Energy, Elsevier, vol. 262(C).
    12. Berecibar, M. & Gandiaga, I. & Villarreal, I. & Omar, N. & Van Mierlo, J. & Van den Bossche, P., 2016. "Critical review of state of health estimation methods of Li-ion batteries for real applications," Renewable and Sustainable Energy Reviews, Elsevier, vol. 56(C), pages 572-587.
    13. Takyi-Aninakwa, Paul & Wang, Shunli & Zhang, Hongying & Yang, Xiao & Fernandez, Carlos, 2023. "A hybrid probabilistic correction model for the state of charge estimation of lithium-ion batteries considering dynamic currents and temperatures," Energy, Elsevier, vol. 273(C).
    14. Lin, Mingqiang & Wu, Denggao & Meng, Jinhao & Wang, Wei & Wu, Ji, 2023. "Health prognosis for lithium-ion battery with multi-feature optimization," Energy, Elsevier, vol. 264(C).
    15. He, Hongwen & Xiong, Rui & Peng, Jiankun, 2016. "Real-time estimation of battery state-of-charge with unscented Kalman filter and RTOS μCOS-II platform," Applied Energy, Elsevier, vol. 162(C), pages 1410-1418.
    16. Ozkurt, Celil & Camci, Fatih & Atamuradov, Vepa & Odorry, Christopher, 2016. "Integration of sampling based battery state of health estimation method in electric vehicles," Applied Energy, Elsevier, vol. 175(C), pages 356-367.
    17. Shahjalal, Mohammad & Roy, Probir Kumar & Shams, Tamanna & Fly, Ashley & Chowdhury, Jahedul Islam & Ahmed, Md. Rishad & Liu, Kailong, 2022. "A review on second-life of Li-ion batteries: prospects, challenges, and issues," Energy, Elsevier, vol. 241(C).
    18. Yang, Fangfang & Li, Weihua & Li, Chuan & Miao, Qiang, 2019. "State-of-charge estimation of lithium-ion batteries based on gated recurrent neural network," Energy, Elsevier, vol. 175(C), pages 66-75.
    19. Yonghong Xu & Cheng Li & Xu Wang & Hongguang Zhang & Fubin Yang & Lili Ma & Yan Wang, 2022. "Joint Estimation Method with Multi-Innovation Unscented Kalman Filter Based on Fractional-Order Model for State of Charge and State of Health Estimation," Sustainability, MDPI, vol. 14(23), pages 1-25, November.
    20. Wang, Yujie & Zhang, Chenbin & Chen, Zonghai, 2014. "A method for joint estimation of state-of-charge and available energy of LiFePO4 batteries," Applied Energy, Elsevier, vol. 135(C), pages 81-87.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:17:y:2023:i:1:p:79-:d:1305634. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.