IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v16y2023i12p4769-d1173029.html
   My bibliography  Save this article

Integrated Demand Response in Multi-Energy Microgrids: A Deep Reinforcement Learning-Based Approach

Author

Listed:
  • Chenhui Xu

    (School of Electrical Engineering, Southeast University, Nanjing 210096, China)

  • Yunkai Huang

    (School of Electrical Engineering, Southeast University, Nanjing 210096, China)

Abstract

The increasing complexity of multi-energy coordinated microgrids presents a challenge for traditional demand response providers to adapt to end users’ multi-energy interactions. The primary aim of demand response providers is to maximize their total profits via designing a pricing strategy for end users. The main challenge lies in the fact that DRPs have no access to the end users’ private preferences. To address this challenge, we propose a deep reinforcement learning-based approach to devise a coordinated scheduling and pricing strategy without requiring any private information. First, we develop an integrated scheduling model that combines power and gas demand response by converting multiple energy sources with different types of residential end users. Then, we formulate the pricing strategy as a Markov Decision Process with an unknown transition. The novel soft actor-critic algorithm is utilized to efficiently train neural networks with the entropy function and to learn the pricing strategies to maximize demand response providers’ profits under various sources of uncertainties. Case studies are conducted to demonstrate the effectiveness of the proposed approach in both deterministic and stochastic environment settings. Our proposed approach is also shown to be effective in handling different levels of uncertainties and achieving the near-optimal pricing strategy.

Suggested Citation

  • Chenhui Xu & Yunkai Huang, 2023. "Integrated Demand Response in Multi-Energy Microgrids: A Deep Reinforcement Learning-Based Approach," Energies, MDPI, vol. 16(12), pages 1-19, June.
  • Handle: RePEc:gam:jeners:v:16:y:2023:i:12:p:4769-:d:1173029
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/16/12/4769/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/1996-1073/16/12/4769/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Tang, Rui & Wang, Shengwei & Li, Hangxin, 2019. "Game theory based interactive demand side management responding to dynamic pricing in price-based demand response of smart grids," Applied Energy, Elsevier, vol. 250(C), pages 118-130.
    2. Natnaree Tubteang & Paramet Wirasanti, 2023. "Peer-to-Peer Electrical Energy Trading Considering Matching Distance and Available Capacity of Distribution Line," Energies, MDPI, vol. 16(6), pages 1-23, March.
    3. Zheng, Ling & Zhou, Bin & Cao, Yijia & Wing Or, Siu & Li, Yong & Wing Chan, Ka, 2022. "Hierarchical distributed multi-energy demand response for coordinated operation of building clusters," Applied Energy, Elsevier, vol. 308(C).
    4. Cao, Wenzhi & Xiao, Jiang-Wen & Cui, Shi-Chang & Liu, Xiao-Kang, 2022. "An efficient and economical storage and energy sharing model for multiple multi-energy microgrids," Energy, Elsevier, vol. 244(PB).
    5. Ali M. Jasim & Basil H. Jasim & Bogdan-Constantin Neagu & Simo Attila, 2023. "Electric Vehicle Battery-Connected Parallel Distribution Generators for Intelligent Demand Management in Smart Microgrids," Energies, MDPI, vol. 16(6), pages 1-29, March.
    6. Yang, Dongfeng & Xu, Yang & Liu, Xiaojun & Jiang, Chao & Nie, Fanjie & Ran, Zixu, 2022. "Economic-emission dispatch problem in integrated electricity and heat system considering multi-energy demand response and carbon capture Technologies," Energy, Elsevier, vol. 253(C).
    7. Wang, Jianxiao & Zhong, Haiwang & Ma, Ziming & Xia, Qing & Kang, Chongqing, 2017. "Review and prospect of integrated demand response in the multi-energy system," Applied Energy, Elsevier, vol. 202(C), pages 772-782.
    8. Vázquez-Canteli, José R. & Nagy, Zoltán, 2019. "Reinforcement learning for demand response: A review of algorithms and modeling techniques," Applied Energy, Elsevier, vol. 235(C), pages 1072-1089.
    9. Venizelos Venizelou & Apostolos C. Tsolakis & Demetres Evagorou & Christos Patsonakis & Ioannis Koskinas & Phivos Therapontos & Lampros Zyglakis & Dimosthenis Ioannidis & George Makrides & Dimitrios T, 2023. "DSO-Aggregator Demand Response Cooperation Framework towards Reliable, Fair and Secure Flexibility Dispatch," Energies, MDPI, vol. 16(6), pages 1-21, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Weiqi Pan & Xiaorong Yu & Zishan Guo & Tao Qian & Yang Li, 2024. "Online EVs Vehicle-to-Grid Scheduling Coordinated with Multi-Energy Microgrids: A Deep Reinforcement Learning-Based Approach," Energies, MDPI, vol. 17(11), pages 1-20, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zheng, Lingwei & Wu, Hao & Guo, Siqi & Sun, Xinyu, 2023. "Real-time dispatch of an integrated energy system based on multi-stage reinforcement learning with an improved action-choosing strategy," Energy, Elsevier, vol. 277(C).
    2. Antonopoulos, Ioannis & Robu, Valentin & Couraud, Benoit & Kirli, Desen & Norbu, Sonam & Kiprakis, Aristides & Flynn, David & Elizondo-Gonzalez, Sergio & Wattam, Steve, 2020. "Artificial intelligence and machine learning approaches to energy demand-side response: A systematic review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 130(C).
    3. Zeng, Lanting & Qiu, Dawei & Sun, Mingyang, 2022. "Resilience enhancement of multi-agent reinforcement learning-based demand response against adversarial attacks," Applied Energy, Elsevier, vol. 324(C).
    4. Zeng, Huibin & Shao, Bilin & Dai, Hongbin & Tian, Ning & Zhao, Wei, 2023. "Incentive-based demand response strategies for natural gas considering carbon emissions and load volatility," Applied Energy, Elsevier, vol. 348(C).
    5. Zhong, Xiaoqing & Zhong, Weifeng & Liu, Yi & Yang, Chao & Xie, Shengli, 2023. "A communication-efficient coalition graph game-based framework for electricity and carbon trading in networked energy hubs," Applied Energy, Elsevier, vol. 329(C).
    6. Lu, Renzhi & Li, Yi-Chang & Li, Yuting & Jiang, Junhui & Ding, Yuemin, 2020. "Multi-agent deep reinforcement learning based demand response for discrete manufacturing systems energy management," Applied Energy, Elsevier, vol. 276(C).
    7. Chen, Sheng & Sun, Guoqiang & Wei, Zhinong & Wang, Dan, 2020. "Dynamic pricing in electricity and natural gas distribution networks: An EPEC model," Energy, Elsevier, vol. 207(C).
    8. Zhang, Kaoshe & Gao, Congchong & Zhang, Gang & Xie, Tuo & Li, Hua, 2024. "Electricity and heat sharing strategy of regional comprehensive energy multi-microgrid based on double-layer game," Energy, Elsevier, vol. 293(C).
    9. Kim, Sunwoo & Choi, Yechan & Park, Joungho & Adams, Derrick & Heo, Seongmin & Lee, Jay H., 2024. "Multi-period, multi-timescale stochastic optimization model for simultaneous capacity investment and energy management decisions for hybrid Micro-Grids with green hydrogen production under uncertainty," Renewable and Sustainable Energy Reviews, Elsevier, vol. 190(PA).
    10. Jin, Ruiyang & Zhou, Yuke & Lu, Chao & Song, Jie, 2022. "Deep reinforcement learning-based strategy for charging station participating in demand response," Applied Energy, Elsevier, vol. 328(C).
    11. Guo, Yurun & Wang, Shugang & Wang, Jihong & Zhang, Tengfei & Ma, Zhenjun & Jiang, Shuang, 2024. "Key district heating technologies for building energy flexibility: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PB).
    12. Shafqat Jawad & Junyong Liu, 2020. "Electrical Vehicle Charging Services Planning and Operation with Interdependent Power Networks and Transportation Networks: A Review of the Current Scenario and Future Trends," Energies, MDPI, vol. 13(13), pages 1-24, July.
    13. Ma, Yixiang & Yu, Lean & Zhang, Guoxing & Lu, Zhiming & Wu, Jiaqian, 2023. "Source-load uncertainty-based multi-objective multi-energy complementary optimal scheduling," Renewable Energy, Elsevier, vol. 219(P1).
    14. Qiu, Rui & Zhang, Haoran & Wang, Guotao & Liang, Yongtu & Yan, Jinyue, 2023. "Green hydrogen-based energy storage service via power-to-gas technologies integrated with multi-energy microgrid," Applied Energy, Elsevier, vol. 350(C).
    15. Wang, Tonghe & Hua, Haochen & Shi, Tianying & Wang, Rui & Sun, Yizhong & Naidoo, Pathmanathan, 2024. "A bi-level dispatch optimization of multi-microgrid considering green electricity consumption willingness under renewable portfolio standard policy," Applied Energy, Elsevier, vol. 356(C).
    16. Tsoumalis, Georgios I. & Bampos, Zafeirios N. & Biskas, Pandelis N. & Keranidis, Stratos D. & Symeonidis, Polychronis A. & Voulgarakis, Dimitrios K., 2022. "A novel system for providing explicit demand response from domestic natural gas boilers," Applied Energy, Elsevier, vol. 317(C).
    17. Gao, Yang & Ai, Qian & He, Xing & Fan, Songli, 2023. "Coordination for regional integrated energy system through target cascade optimization," Energy, Elsevier, vol. 276(C).
    18. Wang, Yi & Qiu, Dawei & Sun, Mingyang & Strbac, Goran & Gao, Zhiwei, 2023. "Secure energy management of multi-energy microgrid: A physical-informed safe reinforcement learning approach," Applied Energy, Elsevier, vol. 335(C).
    19. Gokhale, Gargya & Claessens, Bert & Develder, Chris, 2022. "Physics informed neural networks for control oriented thermal modeling of buildings," Applied Energy, Elsevier, vol. 314(C).
    20. Sun, Hongchang & Niu, Yanlei & Li, Chengdong & Zhou, Changgeng & Zhai, Wenwen & Chen, Zhe & Wu, Hao & Niu, Lanqiang, 2022. "Energy consumption optimization of building air conditioning system via combining the parallel temporal convolutional neural network and adaptive opposition-learning chimp algorithm," Energy, Elsevier, vol. 259(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:16:y:2023:i:12:p:4769-:d:1173029. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.