IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v241y2019icp152-163.html
   My bibliography  Save this article

Power allocation smoothing strategy for hybrid energy storage system based on Markov decision process

Author

Listed:
  • Li, Guidan
  • Yang, Zhe
  • Li, Bin
  • Bi, Huakun

Abstract

The hybrid energy storage system (HESS) in electric vehicle (EV) requires power allocation for optimal performance. Recent researches show that the Markov decision process (MDP) provides promising characteristics for the energy management. However, the power fluctuation, which is rarely considered, can significantly affect the performance of HESS. The paper adopts the bilinear interpolation to smooth the MDP-based strategy. In this way, the power fluctuation can be mitigated, meanwhile the computation cost is not greatly increased. Given the battery and the ultracapacitor, two types of DC/DC converters are employed and the model of the HESS including the battery capacity degradation is introduced. Considering the energy loss and the energy reserve in the HESS, the reward function is built. Utilizing the cumulative reward function, the effect of ultracapacitor pack size on the performance of power allocation is analyzed in detail, the appropriate ultracapacitor size can be obtained further. Simulation results show that MDP strategy with the bilinear interpolation can not only reduce the energy loss by 5–10%, but also prolong the battery cycle life. Finally, the experimental workbench is built. The master-slave strategy is utilized to achieve the power allocation and DC/DC converter control. The proposed strategy is verified by the experimental results.

Suggested Citation

  • Li, Guidan & Yang, Zhe & Li, Bin & Bi, Huakun, 2019. "Power allocation smoothing strategy for hybrid energy storage system based on Markov decision process," Applied Energy, Elsevier, vol. 241(C), pages 152-163.
    • Handle: RePEc:eee:appene:v:241:y:2019:i:c:p:152-163
    DOI: 10.1016/j.apenergy.2019.03.001
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261919303939
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2019.03.001?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Song, Ziyou & Li, Jianqiu & Hou, Jun & Hofmann, Heath & Ouyang, Minggao & Du, Jiuyu, 2018. "The battery-supercapacitor hybrid energy storage system in electric vehicle applications: A case study," Energy, Elsevier, vol. 154(C), pages 433-441.
    2. Wang, Chun & Xiong, Rui & He, Hongwen & Ding, Xiaofeng & Shen, Weixiang, 2016. "Efficiency analysis of a bidirectional DC/DC converter in a hybrid energy storage system for plug-in hybrid electric vehicles," Applied Energy, Elsevier, vol. 183(C), pages 612-622.
    3. Li, Junqiu & Wang, Yihe & Chen, Jianwen & Zhang, Xiaopeng, 2017. "Study on energy management strategy and dynamic modeling for auxiliary power units in range-extended electric vehicles," Applied Energy, Elsevier, vol. 194(C), pages 363-375.
    4. Song, Ziyou & Hofmann, Heath & Li, Jianqiu & Han, Xuebing & Ouyang, Minggao, 2015. "Optimization for a hybrid energy storage system in electric vehicles using dynamic programing approach," Applied Energy, Elsevier, vol. 139(C), pages 151-162.
    5. Raslavičius, Laurencas & Azzopardi, Brian & Keršys, Artūras & Starevičius, Martynas & Bazaras, Žilvinas & Makaras, Rolandas, 2015. "Electric vehicles challenges and opportunities: Lithuanian review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 42(C), pages 786-800.
    6. Zou, Yuan & Liu, Teng & Liu, Dexing & Sun, Fengchun, 2016. "Reinforcement learning-based real-time energy management for a hybrid tracked vehicle," Applied Energy, Elsevier, vol. 171(C), pages 372-382.
    7. Rahmani-Andebili, Mehdi, 2017. "Stochastic, adaptive, and dynamic control of energy storage systems integrated with renewable energy sources for power loss minimization," Renewable Energy, Elsevier, vol. 113(C), pages 1462-1471.
    8. Feroldi, Diego & Carignano, Mauro, 2016. "Sizing for fuel cell/supercapacitor hybrid vehicles based on stochastic driving cycles," Applied Energy, Elsevier, vol. 183(C), pages 645-658.
    9. Xiong, Rui & Cao, Jiayi & Yu, Quanqing, 2018. "Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle," Applied Energy, Elsevier, vol. 211(C), pages 538-548.
    10. Xu, Liangfei & Ouyang, Minggao & Li, Jianqiu & Yang, Fuyuan & Lu, Languang & Hua, Jianfeng, 2013. "Optimal sizing of plug-in fuel cell electric vehicles using models of vehicle performance and system cost," Applied Energy, Elsevier, vol. 103(C), pages 477-487.
    11. Vora, Ashish P. & Jin, Xing & Hoshing, Vaidehi & Saha, Tridib & Shaver, Gregory & Varigonda, Subbarao & Wasynczuk, Oleg & Tyner, Wallace E., 2017. "Design-space exploration of series plug-in hybrid electric vehicles for medium-duty truck applications in a total cost-of-ownership framework," Applied Energy, Elsevier, vol. 202(C), pages 662-672.
    12. Suri, Girish & Onori, Simona, 2016. "A control-oriented cycle-life model for hybrid electric vehicle lithium-ion batteries," Energy, Elsevier, vol. 96(C), pages 644-653.
    13. Huakun Bi & Ping Wang & Zhishuang Wang, 2018. "Common Grounded H-Type Bidirectional DC-DC Converter with a Wide Voltage Conversion Ratio for a Hybrid Energy Storage System," Energies, MDPI, vol. 11(2), pages 1-22, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Wei Guo & Wenyi Fan & Yang Zhao & Jiakun An & Chunguang He & Xiaomei Guo & Yanan Qian & Libo Ma & Hongshan Zhao, 2023. "Grouping Control Strategy for Battery Energy Storage Power Stations Considering the Wind and Solar Power Generation Trend," Energies, MDPI, vol. 16(4), pages 1-15, February.
    2. Yue Liu & Lixin Tian & Zhuyun Xie & Zaili Zhen & Huaping Sun, 2021. "Option to survive or surrender: carbon asset management and optimization in thermal power enterprises from China," Papers 2104.04729, arXiv.org.
    3. Yan, Zhe & Zhang, Yongming & Liang, Runqi & Jin, Wenrui, 2020. "An allocative method of hybrid electrical and thermal energy storage capacity for load shifting based on seasonal difference in district energy planning," Energy, Elsevier, vol. 207(C).
    4. Sekhar Raghu Raman & Ka-Wai (Eric) Cheng & Xiang-Dang Xue & Yat-Chi Fong & Simon Cheung, 2021. "Hybrid Energy Storage System with Vehicle Body Integrated Super-Capacitor and Li-Ion Battery: Model, Design and Implementation, for Distributed Energy Storage," Energies, MDPI, vol. 14(20), pages 1-22, October.
    5. Zhang, Wei & Wang, Jixin & Xu, Zhenyu & Shen, Yuying & Gao, Guangzong, 2022. "A generalized energy management framework for hybrid construction vehicles via model-based reinforcement learning," Energy, Elsevier, vol. 260(C).
    6. Liu, Yue & Tian, Lixin & Sun, Huaping & Zhang, Xiling & Kong, Chuimin, 2022. "Option pricing of carbon asset and its application in digital decision-making of carbon asset," Applied Energy, Elsevier, vol. 310(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wu, Yuankai & Tan, Huachun & Peng, Jiankun & Zhang, Hailong & He, Hongwen, 2019. "Deep reinforcement learning of energy management with continuous control strategy and traffic information for a series-parallel plug-in hybrid electric bus," Applied Energy, Elsevier, vol. 247(C), pages 454-466.
    2. Wang, Chun & Yang, Ruixin & Yu, Quanqing, 2019. "Wavelet transform based energy management strategies for plug-in hybrid electric vehicles considering temperature uncertainty," Applied Energy, Elsevier, vol. 256(C).
    3. Chi T. P. Nguyen & Bảo-Huy Nguyễn & Minh C. Ta & João Pedro F. Trovão, 2023. "Dual-Motor Dual-Source High Performance EV: A Comprehensive Review," Energies, MDPI, vol. 16(20), pages 1-28, October.
    4. Md. Sazal Miah & Molla Shahadat Hossain Lipu & Sheikh Tanzim Meraj & Kamrul Hasan & Shaheer Ansari & Taskin Jamal & Hasan Masrur & Rajvikram Madurai Elavarasan & Aini Hussain, 2021. "Optimized Energy Management Schemes for Electric Vehicle Applications: A Bibliometric Analysis towards Future Trends," Sustainability, MDPI, vol. 13(22), pages 1-38, November.
    5. Xiao, B. & Ruan, J. & Yang, W. & Walker, P.D. & Zhang, N., 2021. "A review of pivotal energy management strategies for extended range electric vehicles," Renewable and Sustainable Energy Reviews, Elsevier, vol. 149(C).
    6. Yang, Weiwei & Ruan, Jiageng & Yang, Jue & Zhang, Nong, 2020. "Investigation of integrated uninterrupted dual input transmission and hybrid energy storage system for electric vehicles," Applied Energy, Elsevier, vol. 262(C).
    7. Yang, Ningkang & Han, Lijin & Xiang, Changle & Liu, Hui & Li, Xunmin, 2021. "An indirect reinforcement learning based real-time energy management strategy via high-order Markov Chain model for a hybrid electric vehicle," Energy, Elsevier, vol. 236(C).
    8. Zhu, Tao & Wills, Richard G.A. & Lot, Roberto & Ruan, Haijun & Jiang, Zhihao, 2021. "Adaptive energy management of a battery-supercapacitor energy storage system for electric vehicles based on flexible perception and neural network fitting," Applied Energy, Elsevier, vol. 292(C).
    9. Lei, Fei & Bai, Yingchun & Zhu, Wenhao & Liu, Jinhong, 2019. "A novel approach for electric powertrain optimization considering vehicle power performance, energy consumption and ride comfort," Energy, Elsevier, vol. 167(C), pages 1040-1050.
    10. Chaofeng Pan & Yanyan Liang & Long Chen & Liao Chen, 2019. "Optimal Control for Hybrid Energy Storage Electric Vehicle to Achieve Energy Saving Using Dynamic Programming Approach," Energies, MDPI, vol. 12(4), pages 1-19, February.
    11. Lian, Renzong & Peng, Jiankun & Wu, Yuankai & Tan, Huachun & Zhang, Hailong, 2020. "Rule-interposing deep reinforcement learning based energy management strategy for power-split hybrid electric vehicle," Energy, Elsevier, vol. 197(C).
    12. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    13. Jiang, Hongliang & Xu, Liangfei & Li, Jianqiu & Hu, Zunyan & Ouyang, Minggao, 2019. "Energy management and component sizing for a fuel cell/battery/supercapacitor hybrid powertrain based on two-dimensional optimization algorithms," Energy, Elsevier, vol. 177(C), pages 386-396.
    14. Desantes, J.M. & Novella, R. & Pla, B. & Lopez-Juarez, M., 2021. "Impact of fuel cell range extender powertrain design on greenhouse gases and NOX emissions in automotive applications," Applied Energy, Elsevier, vol. 302(C).
    15. Song, Ziyou & Li, Jianqiu & Hou, Jun & Hofmann, Heath & Ouyang, Minggao & Du, Jiuyu, 2018. "The battery-supercapacitor hybrid energy storage system in electric vehicle applications: A case study," Energy, Elsevier, vol. 154(C), pages 433-441.
    16. Liu, Xinzhi & Qi, Nanjian & Dai, Keren & Yin, Yajiang & Zhao, Jiahao & Wang, Xiaofeng & You, Zheng, 2022. "Sponge Supercapacitor rule-based energy management strategy for wireless sensor nodes optimized by using dynamic programing algorithm," Energy, Elsevier, vol. 239(PE).
    17. da Silva, Samuel Filgueira & Eckert, Jony Javorski & Corrêa, Fernanda Cristina & Silva, Fabrício Leonardo & Silva, Ludmila C.A. & Dedini, Franco Giuseppe, 2022. "Dual HESS electric vehicle powertrain design and fuzzy control based on multi-objective optimization to increase driving range and battery life cycle," Applied Energy, Elsevier, vol. 324(C).
    18. Shi, Wenzhuo & Huangfu, Yigeng & Xu, Liangcai & Pang, Shengzhao, 2022. "Online energy management strategy considering fuel cell fault for multi-stack fuel cell hybrid vehicle based on multi-agent reinforcement learning," Applied Energy, Elsevier, vol. 328(C).
    19. Zhu, Tao & Wills, Richard G.A. & Lot, Roberto & Kong, Xiaodan & Yan, Xingda, 2021. "Optimal sizing and sensitivity analysis of a battery-supercapacitor energy storage system for electric vehicles," Energy, Elsevier, vol. 221(C).
    20. Alessandro Serpi & Mario Porru, 2019. "Modelling and Design of Real-Time Energy Management Systems for Fuel Cell/Battery Electric Vehicles," Energies, MDPI, vol. 12(22), pages 1-21, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:241:y:2019:i:c:p:152-163. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.