IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v16y2022i1p90-d1010650.html
   My bibliography  Save this article

Handling Computation Hardness and Time Complexity Issue of Battery Energy Storage Scheduling in Microgrids by Deep Reinforcement Learning

Author

Listed:
  • Zeyue Sun

    (School of Electrical Engineering and Telecommunications, University of New South Wales, Sydney, NSW 2052, Australia)

  • Mohsen Eskandari

    (School of Electrical Engineering and Telecommunications, University of New South Wales, Sydney, NSW 2052, Australia)

  • Chaoran Zheng

    (School of Electrical Engineering and Telecommunications, University of New South Wales, Sydney, NSW 2052, Australia)

  • Ming Li

    (School of Electrical Engineering and Telecommunications, University of New South Wales, Sydney, NSW 2052, Australia)

Abstract

With the development of microgrids (MGs), an energy management system (EMS) is required to ensure the stable and economically efficient operation of the MG system. In this paper, an intelligent EMS is proposed by exploiting the deep reinforcement learning (DRL) technique. DRL is employed as the effective method for handling the computation hardness of optimal scheduling of the charge/discharge of battery energy storage in the MG EMS. Since the optimal decision for charge/discharge of the battery depends on its state of charge given from the consecutive time steps, it demands a full-time horizon scheduling to obtain the optimum solution. This, however, increases the time complexity of the EMS and turns it into an NP-hard problem. By considering the energy storage system’s charging/discharging power as the control variable, the DRL agent is trained to investigate the best energy storage control method for both deterministic and stochastic weather scenarios. The efficiency of the strategy suggested in this study in minimizing the cost of purchasing energy is also shown from a quantitative perspective through programming verification and comparison with the results of mixed integer programming and the heuristic genetic algorithm (GA).

Suggested Citation

  • Zeyue Sun & Mohsen Eskandari & Chaoran Zheng & Ming Li, 2022. "Handling Computation Hardness and Time Complexity Issue of Battery Energy Storage Scheduling in Microgrids by Deep Reinforcement Learning," Energies, MDPI, vol. 16(1), pages 1-20, December.
    • Handle: RePEc:gam:jeners:v:16:y:2022:i:1:p:90-:d:1010650
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/16/1/90/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/16/1/90/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Antonopoulos, Ioannis & Robu, Valentin & Couraud, Benoit & Kirli, Desen & Norbu, Sonam & Kiprakis, Aristides & Flynn, David & Elizondo-Gonzalez, Sergio & Wattam, Steve, 2020. "Artificial intelligence and machine learning approaches to energy demand-side response: A systematic review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 130(C).
    2. Muhammad Uzair & Mohsen Eskandari & Li Li & Jianguo Zhu, 2022. "Machine Learning Based Protection Scheme for Low Voltage AC Microgrids," Energies, MDPI, vol. 15(24), pages 1-19, December.
    3. Tabar, Vahid Sohrabi & Jirdehi, Mehdi Ahmadi & Hemmati, Reza, 2017. "Energy management in microgrid based on the multi objective stochastic programming incorporating portable renewable energy resource as demand response option," Energy, Elsevier, vol. 118(C), pages 827-839.
    4. Hua, Haochen & Qin, Yuchao & Hao, Chuantong & Cao, Junwei, 2019. "Optimal energy management strategies for energy Internet via deep reinforcement learning approach," Applied Energy, Elsevier, vol. 239(C), pages 598-609.
    5. Yousef Asadi & Mohsen Eskandari & Milad Mansouri & Andrey V. Savkin & Erum Pathan, 2022. "Frequency and Voltage Control Techniques through Inverter-Interfaced Distributed Energy Resources in Microgrids: A Review," Energies, MDPI, vol. 15(22), pages 1-29, November.
    6. Du, Yan & Zandi, Helia & Kotevska, Olivera & Kurte, Kuldeep & Munk, Jeffery & Amasyali, Kadir & Mckee, Evan & Li, Fangxing, 2021. "Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning," Applied Energy, Elsevier, vol. 281(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Alireza Gorjian & Mohsen Eskandari & Mohammad H. Moradi, 2023. "Conservation Voltage Reduction in Modern Power Systems: Applications, Implementation, Quantification, and AI-Assisted Techniques," Energies, MDPI, vol. 16(5), pages 1-36, March.
    2. Pakeeza Bano & Kashif Imran & Abdul Kashif Janjua & Abdullah Abusorrah & Kinza Fida & Hesham Alhumade, 2023. "System and Market-Wide Impact Analysis of Coordinated Demand Response and Battery Storage Operation by a Load-Serving Entity," Energies, MDPI, vol. 16(4), pages 1-22, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    2. Hernandez-Matheus, Alejandro & Löschenbrand, Markus & Berg, Kjersti & Fuchs, Ida & Aragüés-Peñalba, Mònica & Bullich-Massagué, Eduard & Sumper, Andreas, 2022. "A systematic review of machine learning techniques related to local energy communities," Renewable and Sustainable Energy Reviews, Elsevier, vol. 170(C).
    3. Zhao, Liyuan & Yang, Ting & Li, Wei & Zomaya, Albert Y., 2022. "Deep reinforcement learning-based joint load scheduling for household multi-energy system," Applied Energy, Elsevier, vol. 324(C).
    4. Homod, Raad Z. & Togun, Hussein & Kadhim Hussein, Ahmed & Noraldeen Al-Mousawi, Fadhel & Yaseen, Zaher Mundher & Al-Kouz, Wael & Abd, Haider J. & Alawi, Omer A. & Goodarzi, Marjan & Hussein, Omar A., 2022. "Dynamics analysis of a novel hybrid deep clustering for unsupervised learning by reinforcement of multi-agent to energy saving in intelligent buildings," Applied Energy, Elsevier, vol. 313(C).
    5. Alireza Gorjian & Mohsen Eskandari & Mohammad H. Moradi, 2023. "Conservation Voltage Reduction in Modern Power Systems: Applications, Implementation, Quantification, and AI-Assisted Techniques," Energies, MDPI, vol. 16(5), pages 1-36, March.
    6. Zhang, Bin & Hu, Weihao & Xu, Xiao & Li, Tao & Zhang, Zhenyuan & Chen, Zhe, 2022. "Physical-model-free intelligent energy management for a grid-connected hybrid wind-microturbine-PV-EV energy system via deep reinforcement learning approach," Renewable Energy, Elsevier, vol. 200(C), pages 433-448.
    7. Panagiotis Michailidis & Iakovos Michailidis & Dimitrios Vamvakas & Elias Kosmatopoulos, 2023. "Model-Free HVAC Control in Buildings: A Review," Energies, MDPI, vol. 16(20), pages 1-45, October.
    8. Nyong-Bassey, Bassey Etim & Giaouris, Damian & Patsios, Charalampos & Papadopoulou, Simira & Papadopoulos, Athanasios I. & Walker, Sara & Voutetakis, Spyros & Seferlis, Panos & Gadoue, Shady, 2020. "Reinforcement learning based adaptive power pinch analysis for energy management of stand-alone hybrid energy storage systems considering uncertainty," Energy, Elsevier, vol. 193(C).
    9. Fuster-Palop, Enrique & Prades-Gil, Carlos & Masip, X. & Viana-Fons, Joan D. & Payá, Jorge, 2021. "Innovative regression-based methodology to assess the techno-economic performance of photovoltaic installations in urban areas," Renewable and Sustainable Energy Reviews, Elsevier, vol. 149(C).
    10. Davide Coraci & Silvio Brandi & Marco Savino Piscitelli & Alfonso Capozzoli, 2021. "Online Implementation of a Soft Actor-Critic Agent to Enhance Indoor Temperature Control and Energy Efficiency in Buildings," Energies, MDPI, vol. 14(4), pages 1-26, February.
    11. Zhu, Jiaoyiling & Hu, Weihao & Xu, Xiao & Liu, Haoming & Pan, Li & Fan, Haoyang & Zhang, Zhenyuan & Chen, Zhe, 2022. "Optimal scheduling of a wind energy dominated distribution network via a deep reinforcement learning approach," Renewable Energy, Elsevier, vol. 201(P1), pages 792-801.
    12. Jingjing Zhai & Xiaobei Wu & Zihao Li & Shaojie Zhu & Bo Yang & Haoming Liu, 2021. "Day-Ahead and Intra-Day Collaborative Optimized Operation among Multiple Energy Stations," Energies, MDPI, vol. 14(4), pages 1-33, February.
    13. Zhang, Yijie & Ma, Tao & Elia Campana, Pietro & Yamaguchi, Yohei & Dai, Yanjun, 2020. "A techno-economic sizing method for grid-connected household photovoltaic battery systems," Applied Energy, Elsevier, vol. 269(C).
    14. Ahmad, Tanveer & Chen, Huanxin, 2019. "Deep learning for multi-scale smart energy forecasting," Energy, Elsevier, vol. 175(C), pages 98-112.
    15. Ahmad, Tanveer & Madonski, Rafal & Zhang, Dongdong & Huang, Chao & Mujeeb, Asad, 2022. "Data-driven probabilistic machine learning in sustainable smart energy/smart energy systems: Key developments, challenges, and future research opportunities in the context of smart grid paradigm," Renewable and Sustainable Energy Reviews, Elsevier, vol. 160(C).
    16. Sijia Li & Arman Oshnoei & Frede Blaabjerg & Amjad Anvari-Moghaddam, 2023. "Hierarchical Control for Microgrids: A Survey on Classical and Machine Learning-Based Methods," Sustainability, MDPI, vol. 15(11), pages 1-22, June.
    17. Zhang, Bin & Hu, Weihao & Ghias, Amer M.Y.M. & Xu, Xiao & Chen, Zhe, 2022. "Multi-agent deep reinforcement learning-based coordination control for grid-aware multi-buildings," Applied Energy, Elsevier, vol. 328(C).
    18. M. Usman Saleem & Mustafa Shakir & M. Rehan Usman & M. Hamza Tahir Bajwa & Noman Shabbir & Payam Shams Ghahfarokhi & Kamran Daniel, 2023. "Integrating Smart Energy Management System with Internet of Things and Cloud Computing for Efficient Demand Side Management in Smart Grids," Energies, MDPI, vol. 16(12), pages 1-21, June.
    19. Gabriella Ferruzzi & Giorgio Graditi & Federico Rossi, 2020. "A joint approach for strategic bidding of a microgrid in energy and spinning reserve markets," Energy & Environment, , vol. 31(1), pages 88-115, February.
    20. Patrick Sunday Onen & Geev Mokryani & Rana H. A. Zubo, 2022. "Planning of Multi-Vector Energy Systems with High Penetration of Renewable Energy Source: A Comprehensive Review," Energies, MDPI, vol. 15(15), pages 1-25, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:16:y:2022:i:1:p:90-:d:1010650. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.