IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v229y2021ics0360544221009531.html
   My bibliography  Save this article

Energy consumption and battery aging minimization using a Q-learning strategy for a battery/ultracapacitor electric vehicle

Author

Listed:
  • Xu, Bin
  • Shi, Junzhe
  • Li, Sixu
  • Li, Huayi
  • Wang, Zhe

Abstract

Propulsion system electrification revolution has been undergoing in the automotive industry. The electrified propulsion system improves energy efficiency and reduces the dependence on fossil fuel. However, the batteries of electric vehicles experience degradation process during vehicle operation. Research considering both battery degradation and energy consumption in battery/ultracapacitor electric vehicles is still lacking. This study proposes a Q-learning-based strategy to minimize battery degradation and energy consumption. Besides Q-learning, two rule-based energy management methods are also proposed and optimized using Particle Swarm Optimization algorithm. A vehicle propulsion system model is first presented, where the severity factor battery degradation model is considered and experimentally validated with the help of Genetic Algorithm. In the results analysis, Q-learning is first explained with the optimal policy map after learning. Then, the result from a vehicle without ultracapacitor is used as the baseline, which is compared with the results from the vehicle with ultracapacitor using Q-learning, and two rule-based methods as the energy management strategies. At the learning and validation driving cycles, the results indicate that the Q-learning strategy slows down the battery degradation by 13–20% and increases the vehicle range by 1.5–2% compared with the baseline vehicle without ultracapacitor.

Suggested Citation

  • Xu, Bin & Shi, Junzhe & Li, Sixu & Li, Huayi & Wang, Zhe, 2021. "Energy consumption and battery aging minimization using a Q-learning strategy for a battery/ultracapacitor electric vehicle," Energy, Elsevier, vol. 229(C).
    • Handle: RePEc:eee:energy:v:229:y:2021:i:c:s0360544221009531
    DOI: 10.1016/j.energy.2021.120705
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544221009531
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2021.120705?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Fan Yang & Yuanyuan Xie & Yelin Deng & Chris Yuan, 2018. "Predictive modeling of battery degradation and greenhouse gas emissions from U.S. state-level electric vehicle operation," Nature Communications, Nature, vol. 9(1), pages 1-10, December.
    2. Xie, Yunkun & Li, Yangyang & Zhao, Zhichao & Dong, Hao & Wang, Shuqian & Liu, Jingping & Guan, Jinhuan & Duan, Xiongbo, 2020. "Microsimulation of electric vehicle energy consumption and driving range," Applied Energy, Elsevier, vol. 267(C).
    3. Kim, Youngki & Figueroa-Santos, Miriam & Prakash, Niket & Baek, Stanley & Siegel, Jason B. & Rizzo, Denise M., 2020. "Co-optimization of speed trajectory and power management for a fuel-cell/battery electric vehicle," Applied Energy, Elsevier, vol. 260(C).
    4. Xiong, Rui & Duan, Yanzhou & Cao, Jiayi & Yu, Quanqing, 2018. "Battery and ultracapacitor in-the-loop approach to validate a real-time power management method for an all-climate electric vehicle," Applied Energy, Elsevier, vol. 217(C), pages 153-165.
    5. Cong Zhang & Dai Wang & Bin Wang & Fan Tong, 2020. "Battery Degradation Minimization-Oriented Hybrid Energy Storage System for Electric Vehicles," Energies, MDPI, vol. 13(1), pages 1-21, January.
    6. Xu, Bin & Rathod, Dhruvang & Zhang, Darui & Yebi, Adamu & Zhang, Xueyu & Li, Xiaoya & Filipi, Zoran, 2020. "Parametric study on reinforcement learning optimized energy management strategy for a hybrid electric vehicle," Applied Energy, Elsevier, vol. 259(C).
    7. Arumugam Manthiram, 2020. "A reflection on lithium-ion battery cathode chemistry," Nature Communications, Nature, vol. 11(1), pages 1-9, December.
    8. Suri, Girish & Onori, Simona, 2016. "A control-oriented cycle-life model for hybrid electric vehicle lithium-ion batteries," Energy, Elsevier, vol. 96(C), pages 644-653.
    9. Pelletier, Samuel & Jabali, Ola & Laporte, Gilbert & Veneroni, Marco, 2017. "Battery degradation and behaviour for electric vehicles: Review and numerical analyses of several models," Transportation Research Part B: Methodological, Elsevier, vol. 103(C), pages 158-187.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Liu, Qin & Zhang, Wencan & Zhang, Zhongbo & Qin, Qichao, 2022. "A drive system global control strategy for electric vehicle based on optimized acceleration curve," Energy, Elsevier, vol. 248(C).
    2. Caulfield, Brian & Furszyfer, Dylan & Stefaniec, Agnieszka & Foley, Aoife, 2022. "Measuring the equity impacts of government subsidies for electric vehicles," Energy, Elsevier, vol. 248(C).
    3. Harri Aaltonen & Seppo Sierla & Rakshith Subramanya & Valeriy Vyatkin, 2021. "A Simulation Environment for Training a Reinforcement Learning Agent Trading a Battery Storage," Energies, MDPI, vol. 14(17), pages 1-20, September.
    4. Du, Guodong & Zou, Yuan & Zhang, Xudong & Guo, Lingxiong & Guo, Ningyuan, 2022. "Energy management for a hybrid electric vehicle based on prioritized deep reinforcement learning framework," Energy, Elsevier, vol. 241(C).
    5. Guo, Xiaokai & Yan, Xianguo & Chen, Zhi & Meng, Zhiyu, 2022. "Research on energy management strategy of heavy-duty fuel cell hybrid vehicles based on dueling-double-deep Q-network," Energy, Elsevier, vol. 260(C).
    6. Awab Baqar & Mamadou Baïlo Camara & Brayima Dakyo, 2022. "Energy Management in the Multi-Source Systems," Energies, MDPI, vol. 15(8), pages 1-4, April.
    7. Mokesioluwa Fanoro & Mladen Božanić & Saurabh Sinha, 2022. "A Review of the Impact of Battery Degradation on Energy Management Systems with a Special Emphasis on Electric Vehicles," Energies, MDPI, vol. 15(16), pages 1-29, August.
    8. Andrea Di Martino & Seyed Mahdi Miraftabzadeh & Michela Longo, 2022. "Strategies for the Modelisation of Electric Vehicle Energy Consumption: A Review," Energies, MDPI, vol. 15(21), pages 1-20, October.
    9. Bo, Lin & Han, Lijin & Xiang, Changle & Liu, Hui & Ma, Tian, 2022. "A Q-learning fuzzy inference system based online energy management strategy for off-road hybrid electric vehicles," Energy, Elsevier, vol. 252(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhao, Yang & Wang, Zhenpo & Shen, Zuo-Jun Max & Zhang, Lei & Dorrell, David G. & Sun, Fengchun, 2022. "Big data-driven decoupling framework enabling quantitative assessments of electric vehicle performance degradation," Applied Energy, Elsevier, vol. 327(C).
    2. Wang, Hua & Zhao, De & Meng, Qiang & Ong, Ghim Ping & Lee, Der-Horng, 2020. "Network-level energy consumption estimation for electric vehicles considering vehicle and user heterogeneity," Transportation Research Part A: Policy and Practice, Elsevier, vol. 132(C), pages 30-46.
    3. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    4. Wang, Hua & Zhao, De & Cai, Yutong & Meng, Qiang & Ong, Ghim Ping, 2021. "Taxi trajectory data based fast-charging facility planning for urban electric taxi systems," Applied Energy, Elsevier, vol. 286(C).
    5. Sarvaiya, Shradhdha & Ganesh, Sachin & Xu, Bin, 2021. "Comparative analysis of hybrid vehicle energy management strategies with optimization of fuel economy and battery life," Energy, Elsevier, vol. 228(C).
    6. Rauf, Huzaifa & Khalid, Muhammad & Arshad, Naveed, 2022. "Machine learning in state of health and remaining useful life estimation: Theoretical and technological development in battery degradation modelling," Renewable and Sustainable Energy Reviews, Elsevier, vol. 156(C).
    7. Shi, Dehua & Liu, Sheng & Cai, Yingfeng & Wang, Shaohua & Li, Haoran & Chen, Long, 2021. "Pontryagin’s minimum principle based fuzzy adaptive energy management for hybrid electric vehicle using real-time traffic information," Applied Energy, Elsevier, vol. 286(C).
    8. Farmann, Alexander & Waag, Wladislaw & Sauer, Dirk Uwe, 2016. "Application-specific electrical characterization of high power batteries with lithium titanate anodes for electric vehicles," Energy, Elsevier, vol. 112(C), pages 294-306.
    9. He, Qiang & Yang, Yang & Luo, Chang & Zhai, Jun & Luo, Ronghua & Fu, Chunyun, 2022. "Energy recovery strategy optimization of dual-motor drive electric vehicle based on braking safety and efficient recovery," Energy, Elsevier, vol. 248(C).
    10. Zhou, Yu & Meng, Qiang & Ong, Ghim Ping, 2022. "Electric Bus Charging Scheduling for a Single Public Transport Route Considering Nonlinear Charging Profile and Battery Degradation Effect," Transportation Research Part B: Methodological, Elsevier, vol. 159(C), pages 49-75.
    11. Sun, Alexander Y., 2020. "Optimal carbon storage reservoir management through deep reinforcement learning," Applied Energy, Elsevier, vol. 278(C).
    12. Barelli, L. & Bidini, G. & Bonucci, F. & Castellini, L. & Fratini, A. & Gallorini, F. & Zuccari, A., 2019. "Flywheel hybridization to improve battery life in energy storage systems coupled to RES plants," Energy, Elsevier, vol. 173(C), pages 937-950.
    13. Eckert, Jony Javorski & Silva, Fabrício L. & da Silva, Samuel Filgueira & Bueno, André Valente & de Oliveira, Mona Lisa Moura & Silva, Ludmila C.A., 2022. "Optimal design and power management control of hybrid biofuel–electric powertrain," Applied Energy, Elsevier, vol. 325(C).
    14. Matteo Acquarone & Claudio Maino & Daniela Misul & Ezio Spessa & Antonio Mastropietro & Luca Sorrentino & Enrico Busto, 2023. "Influence of the Reward Function on the Selection of Reinforcement Learning Agents for Hybrid Electric Vehicles Real-Time Control," Energies, MDPI, vol. 16(6), pages 1-22, March.
    15. Yang, Ningkang & Han, Lijin & Xiang, Changle & Liu, Hui & Li, Xunmin, 2021. "An indirect reinforcement learning based real-time energy management strategy via high-order Markov Chain model for a hybrid electric vehicle," Energy, Elsevier, vol. 236(C).
    16. Nan, Sirui & Tu, Ran & Li, Tiezhu & Sun, Jian & Chen, Haibo, 2022. "From driving behavior to energy consumption: A novel method to predict the energy consumption of electric bus," Energy, Elsevier, vol. 261(PA).
    17. Huang, Ruchen & He, Hongwen & Gao, Miaojue, 2023. "Training-efficient and cost-optimal energy management for fuel cell hybrid electric bus based on a novel distributed deep reinforcement learning framework," Applied Energy, Elsevier, vol. 346(C).
    18. Duggal, Angel Swastik & Singh, Rajesh & Gehlot, Anita & Gupta, Lovi Raj & Akram, Sheik Vaseem & Prakash, Chander & Singh, Sunpreet & Kumar, Raman, 2021. "Infrastructure, mobility and safety 4.0: Modernization in road transportation," Technology in Society, Elsevier, vol. 67(C).
    19. Yang Xia & Wenjia Zeng & Xinjie Xing & Yuanzhu Zhan & Kim Hua Tan & Ajay Kumar, 2023. "Joint optimisation of drone routing and battery wear for sustainable supply chain development: a mixed-integer programming model based on blockchain-enabled fleet sharing," Annals of Operations Research, Springer, vol. 327(1), pages 89-127, August.
    20. Zhu, Tao & Wills, Richard G.A. & Lot, Roberto & Ruan, Haijun & Jiang, Zhihao, 2021. "Adaptive energy management of a battery-supercapacitor energy storage system for electric vehicles based on flexible perception and neural network fitting," Applied Energy, Elsevier, vol. 292(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:229:y:2021:i:c:s0360544221009531. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.