IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v17y2024i21p5425-d1510400.html
   My bibliography  Save this article

Reinforcement Learning-Enhanced Adaptive Scheduling of Battery Energy Storage Systems in Energy Markets

Author

Listed:
  • Yang Liu

    (Electric Power Dispatching and Control Center of Guangdong Power Grid Co., Ltd., Guangzhou 510220, China)

  • Qiuyu Lu

    (Electric Power Dispatching and Control Center of Guangdong Power Grid Co., Ltd., Guangzhou 510220, China)

  • Zhenfan Yu

    (Electric Power Dispatching and Control Center of Guangdong Power Grid Co., Ltd., Guangzhou 510220, China)

  • Yue Chen

    (Electric Power Dispatching and Control Center of Guangdong Power Grid Co., Ltd., Guangzhou 510220, China)

  • Yinguo Yang

    (Electric Power Dispatching and Control Center of Guangdong Power Grid Co., Ltd., Guangzhou 510220, China)

Abstract

Battery Energy Storage Systems (BESSs) play a vital role in modern power grids by optimally dispatching energy according to the price signal. This paper proposes a reinforcement learning-based model that optimizes BESS scheduling with the proposed Q-learning algorithm combined with an epsilon-greedy strategy. The proposed epsilon-greedy strategy-based Q-learning algorithm can efficiently manage energy dispatching under uncertain price signals and multi-day operations without retraining. Simulations are conducted under different scenarios, considering electricity price fluctuations and battery aging conditions. Results show that the proposed algorithm demonstrates enhanced economic returns and adaptability compared to traditional methods, providing a practical solution for intelligent BESS scheduling that supports grid stability and the efficient use of renewable energy.

Suggested Citation

  • Yang Liu & Qiuyu Lu & Zhenfan Yu & Yue Chen & Yinguo Yang, 2024. "Reinforcement Learning-Enhanced Adaptive Scheduling of Battery Energy Storage Systems in Energy Markets," Energies, MDPI, vol. 17(21), pages 1-17, October.
    • Handle: RePEc:gam:jeners:v:17:y:2024:i:21:p:5425-:d:1510400
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/17/21/5425/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/17/21/5425/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Wu, Peng & Partridge, Julius & Bucknall, Richard, 2020. "Cost-effective reinforcement learning energy management for plug-in hybrid fuel cell and battery ships," Applied Energy, Elsevier, vol. 275(C).
    2. Meng, Jinhao & Cai, Lei & Stroe, Daniel-Ioan & Ma, Junpeng & Luo, Guangzhao & Teodorescu, Remus, 2020. "An optimized ensemble learning framework for lithium-ion Battery State of Health estimation in energy storage system," Energy, Elsevier, vol. 206(C).
    3. Chen, Xi & Liu, Zhongbing & Wang, Pengcheng & Li, Benjia & Liu, Ruimiao & Zhang, Ling & Zhao, Chengliang & Luo, Songqin, 2023. "Multi-objective optimization of battery capacity of grid-connected PV-BESS system in hybrid building energy sharing community considering time-of-use tariff," Applied Energy, Elsevier, vol. 350(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wei, Jingwen & Chen, Chunlin, 2021. "A multi-timescale framework for state monitoring and lifetime prognosis of lithium-ion batteries," Energy, Elsevier, vol. 229(C).
    2. Ziad Ragab & Ehsan Pashajavid & Sumedha Rajakaruna, 2024. "Optimal Sizing and Economic Analysis of Community Battery Systems Considering Sensitivity and Uncertainty Factors," Energies, MDPI, vol. 17(18), pages 1-20, September.
    3. Behzadi, Amirmohammad & Duwig, Christophe & Ploskic, Adnan & Holmberg, Sture & Sadrizadeh, Sasan, 2024. "Application to novel smart techniques for decarbonization of commercial building heating and cooling through optimal energy management," Applied Energy, Elsevier, vol. 376(PA).
    4. Liu, Hanyou & Fan, Ailong & Li, Yongping & Bucknall, Richard & Chen, Li, 2024. "Hierarchical distributed MPC method for hybrid energy management: A case study of ship with variable operating conditions," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PA).
    5. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    6. Maja Perčić & Nikola Vladimir & Marija Koričan, 2021. "Electrification of Inland Waterway Ships Considering Power System Lifetime Emissions and Costs," Energies, MDPI, vol. 14(21), pages 1-25, October.
    7. Elkholy, M.H. & Metwally, Hamid & Farahat, M.A. & Senjyu, Tomonobu & Elsayed Lotfy, Mohammed, 2022. "Smart centralized energy management system for autonomous microgrid using FPGA," Applied Energy, Elsevier, vol. 317(C).
    8. Yang, Zirong & Jiao, Kui & Wu, Kangcheng & Shi, Weilong & Jiang, Shangfeng & Zhang, Longhai & Du, Qing, 2021. "Numerical investigations of assisted heating cold start strategies for proton exchange membrane fuel cell systems," Energy, Elsevier, vol. 222(C).
    9. Shi, Mingjie & Xu, Jun & Lin, Chuanping & Mei, Xuesong, 2022. "A fast state-of-health estimation method using single linear feature for lithium-ion batteries," Energy, Elsevier, vol. 256(C).
    10. Touzani, Samir & Prakash, Anand Krishnan & Wang, Zhe & Agarwal, Shreya & Pritoni, Marco & Kiran, Mariam & Brown, Richard & Granderson, Jessica, 2021. "Controlling distributed energy resources via deep reinforcement learning for load flexibility and energy efficiency," Applied Energy, Elsevier, vol. 304(C).
    11. Xie, Peilin & Tan, Sen & Bazmohammadi, Najmeh & Guerrero, Josep. M. & Vasquez, Juan. C. & Alcala, Jose Matas & Carreño, Jorge El Mariachet, 2022. "A distributed real-time power management scheme for shipboard zonal multi-microgrid system," Applied Energy, Elsevier, vol. 317(C).
    12. Chen, Chunyu & Cui, Mingjian & Fang, Xin & Ren, Bixing & Chen, Yang, 2020. "Load altering attack-tolerant defense strategy for load frequency control system," Applied Energy, Elsevier, vol. 280(C).
    13. Sui, Xin & He, Shan & Vilsen, Søren B. & Meng, Jinhao & Teodorescu, Remus & Stroe, Daniel-Ioan, 2021. "A review of non-probabilistic machine learning-based state of health estimation techniques for Lithium-ion battery," Applied Energy, Elsevier, vol. 300(C).
    14. Yong Zhu & Mingyi Liu & Lin Wang & Jianxing Wang, 2022. "Potential Failure Prediction of Lithium-ion Battery Energy Storage System by Isolation Density Method," Sustainability, MDPI, vol. 14(12), pages 1-14, June.
    15. Macias, A. & Kandidayeni, M. & Boulon, L. & Trovão, J.P., 2021. "Fuel cell-supercapacitor topologies benchmark for a three-wheel electric vehicle powertrain," Energy, Elsevier, vol. 224(C).
    16. Lyu, Zhiqiang & Wang, Geng & Gao, Renjing, 2022. "Synchronous state of health estimation and remaining useful lifetime prediction of Li-Ion battery through optimized relevance vector machine framework," Energy, Elsevier, vol. 251(C).
    17. Tsoumpris, Charalampos & Theotokatos, Gerasimos, 2023. "A decision-making approach for the health-aware energy management of ship hybrid power plants," Reliability Engineering and System Safety, Elsevier, vol. 235(C).
    18. Song, Tiewei & Fu, Lijun & Zhong, Linlin & Fan, Yaxiang & Shang, Qianyi, 2024. "HP3O algorithm-based all electric ship energy management strategy integrating demand-side adjustment," Energy, Elsevier, vol. 295(C).
    19. Xu, Zhicheng & Wang, Jun & Lund, Peter D. & Zhang, Yaoming, 2021. "Estimation and prediction of state of health of electric vehicle batteries using discrete incremental capacity analysis based on real driving data," Energy, Elsevier, vol. 225(C).
    20. Jonas Quernheim & Eberhard Waffenschmidt, 2024. "A Comparative Evaluation of Community-Used District and Individual Battery Storage Systems for Photovoltaic Energy Systems," Energies, MDPI, vol. 17(17), pages 1-12, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:17:y:2024:i:21:p:5425-:d:1510400. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.