IDEAS home Printed from https://ideas.repec.org/a/gam/jeners/v15y2022i21p7866-d951269.html
   My bibliography  Save this article

System Frequency Control Method Driven by Deep Reinforcement Learning and Customer Satisfaction for Thermostatically Controlled Load

Author

Listed:
  • Rusi Chen

    (State Grid Hubei Electric Power Research Institute, Wuhan 430077, China)

  • Haiguang Liu

    (State Grid Hubei Electric Power Research Institute, Wuhan 430077, China)

  • Chengquan Liu

    (Department of Electrical Engineering, Shanghai University of Electric Power, Shanghai 201306, China)

  • Guangzheng Yu

    (Department of Electrical Engineering, Shanghai University of Electric Power, Shanghai 201306, China)

  • Xuan Yang

    (State Grid Hubei Electric Power Co., Ltd., Wuhan 430072, China)

  • Yue Zhou

    (State Grid Hubei Electric Power Co., Ltd., Wuhan 430072, China)

Abstract

The intermittence and fluctuation of renewable energy aggravate the power fluctuation of the power grid and pose a severe challenge to the frequency stability of the power system. Thermostatically controlled loads can participate in the frequency regulation of the power grid due to their flexibility. Aiming to solve the problem of the traditional control methods, which have limited adjustment ability, and to have a positive influence on customers, a deep reinforcement learning control strategy based on the framework of soft actor–critic is proposed, considering customer satisfaction. Firstly, the energy storage index and the discomfort index of different users are defined. Secondly, the fuzzy comprehensive evaluation method is applied to evaluate customer satisfaction. Then, the multi-agent models of thermostatically controlled loads are established based on the soft actor–critic algorithm. The models are trained by using the local information of thermostatically controlled loads, and the comprehensive evaluation index fed back by users and the frequency deviation. After training, each agent can realize the cooperative response of thermostatically controlled loads to the system frequency only by relying on the local information. The simulation results show that the proposed strategy can not only reduce the frequency fluctuation, but also improve customer satisfaction.

Suggested Citation

  • Rusi Chen & Haiguang Liu & Chengquan Liu & Guangzheng Yu & Xuan Yang & Yue Zhou, 2022. "System Frequency Control Method Driven by Deep Reinforcement Learning and Customer Satisfaction for Thermostatically Controlled Load," Energies, MDPI, vol. 15(21), pages 1-19, October.
    • Handle: RePEc:gam:jeners:v:15:y:2022:i:21:p:7866-:d:951269
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1996-1073/15/21/7866/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1996-1073/15/21/7866/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Zexu Chen & Jing Shi & Zhaofang Song & Wangwang Yang & Zitong Zhang, 2022. "Genetic Algorithm Based Temperature-Queuing Method for Aggregated IAC Load Control," Energies, MDPI, vol. 15(2), pages 1-16, January.
    2. Pallonetto, Fabiano & De Rosa, Mattia & Milano, Federico & Finn, Donal P., 2019. "Demand response algorithms for smart-grid ready residential buildings using machine learning models," Applied Energy, Elsevier, vol. 239(C), pages 1265-1282.
    3. Rongxiang Zhang & Xiaodong Chu & Wen Zhang & Yutian Liu, 2015. "Active Participation of Air Conditioners in Power System Frequency Control Considering Users’ Thermal Comfort," Energies, MDPI, vol. 8(10), pages 1-24, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nallapaneni Manoj Kumar & Aneesh A. Chand & Maria Malvoni & Kushal A. Prasad & Kabir A. Mamun & F.R. Islam & Shauhrat S. Chopra, 2020. "Distributed Energy Resources and the Application of AI, IoT, and Blockchain in Smart Grids," Energies, MDPI, vol. 13(21), pages 1-42, November.
    2. Chen, Yuxin & Sun, Yongjun & Yang, Jinling & Tan, Jiaqi & Liu, Yang & Gao, Dian-ce, 2024. "Demand response with PCM-based pipe-embedded wall in commercial buildings: Combined passive and active energy storage in envelopes," Energy, Elsevier, vol. 308(C).
    3. Oleh Lukianykhin & Tetiana Bogodorova, 2021. "Voltage Control-Based Ancillary Service Using Deep Reinforcement Learning," Energies, MDPI, vol. 14(8), pages 1-22, April.
    4. Ibrahim, Muhammad Sohail & Dong, Wei & Yang, Qiang, 2020. "Machine learning driven smart electric power systems: Current trends and new perspectives," Applied Energy, Elsevier, vol. 272(C).
    5. Xu, Fangyuan & Zhu, Weidong & Wang, Yi Fei & Lai, Chun Sing & Yuan, Haoliang & Zhao, Yujia & Guo, Siming & Fu, Zhengxin, 2022. "A new deregulated demand response scheme for load over-shifting city in regulated power market," Applied Energy, Elsevier, vol. 311(C).
    6. Ran, Fengming & Gao, Dian-ce & Zhang, Xu & Chen, Shuyue, 2020. "A virtual sensor based self-adjusting control for HVAC fast demand response in commercial buildings towards smart grid applications," Applied Energy, Elsevier, vol. 269(C).
    7. Máximo A. Domínguez-Garabitos & Víctor S. Ocaña-Guevara & Félix Santos-García & Adriana Arango-Manrique & Miguel Aybar-Mejía, 2022. "A Methodological Proposal for Implementing Demand-Shifting Strategies in the Wholesale Electricity Market," Energies, MDPI, vol. 15(4), pages 1-28, February.
    8. Lu, Qing & Zhang, Yufeng, 2022. "A multi-objective optimization model considering users' satisfaction and multi-type demand response in dynamic electricity price," Energy, Elsevier, vol. 240(C).
    9. Das, Laya & Garg, Dinesh & Srinivasan, Babji, 2020. "NeuralCompression: A machine learning approach to compress high frequency measurements in smart grid," Applied Energy, Elsevier, vol. 257(C).
    10. Siyue Lu & Baoqun Zhang & Longfei Ma & Hui Xu & Yuantong Li & Shaobing Yang, 2023. "Economic Load-Reduction Strategy of Central Air Conditioning Based on Convolutional Neural Network and Pre-Cooling," Energies, MDPI, vol. 16(13), pages 1-22, June.
    11. Lilia Tightiz & Joon Yoo, 2022. "A Review on a Data-Driven Microgrid Management System Integrating an Active Distribution Network: Challenges, Issues, and New Trends," Energies, MDPI, vol. 15(22), pages 1-24, November.
    12. Arsalan Masood & Ubaid Ahmed & Syed Zulqadar Hassan & Ahsan Raza Khan & Anzar Mahmood, 2025. "Economic Value Creation of Artificial Intelligence in Supporting Variable Renewable Energy Resource Integration to Power Systems: A Systematic Review," Sustainability, MDPI, vol. 17(6), pages 1-42, March.
    13. Ottavia Valentini & Nikoleta Andreadou & Paolo Bertoldi & Alexandre Lucas & Iolanda Saviuc & Evangelos Kotsakis, 2022. "Demand Response Impact Evaluation: A Review of Methods for Estimating the Customer Baseline Load," Energies, MDPI, vol. 15(14), pages 1-36, July.
    14. Zhan, Sicheng & Chong, Adrian, 2021. "Data requirements and performance evaluation of model predictive control in buildings: A modeling perspective," Renewable and Sustainable Energy Reviews, Elsevier, vol. 142(C).
    15. Vidya Krishnan Mololoth & Saguna Saguna & Christer Åhlund, 2023. "Blockchain and Machine Learning for Future Smart Grids: A Review," Energies, MDPI, vol. 16(1), pages 1-39, January.
    16. Lu, Renzhi & Li, Yi-Chang & Li, Yuting & Jiang, Junhui & Ding, Yuemin, 2020. "Multi-agent deep reinforcement learning based demand response for discrete manufacturing systems energy management," Applied Energy, Elsevier, vol. 276(C).
    17. Pallonetto, Fabiano & De Rosa, Mattia & D’Ettorre, Francesco & Finn, Donal P., 2020. "On the assessment and control optimisation of demand response programs in residential buildings," Renewable and Sustainable Energy Reviews, Elsevier, vol. 127(C).
    18. Wakui, Tetsuya & Sawada, Kento & Yokoyama, Ryohei & Aki, Hirohisa, 2019. "Predictive management for energy supply networks using photovoltaics, heat pumps, and battery by two-stage stochastic programming and rule-based control," Energy, Elsevier, vol. 179(C), pages 1302-1319.
    19. Mattia De Rosa & Marcus Brennenstuhl & Carlos Andrade Cabrera & Ursula Eicker & Donal P. Finn, 2019. "An Iterative Methodology for Model Complexity Reduction in Residential Building Simulation," Energies, MDPI, vol. 12(12), pages 1-20, June.
    20. Ying-Yi Hong, 2016. "Electric Power Systems Research," Energies, MDPI, vol. 9(10), pages 1-4, October.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jeners:v:15:y:2022:i:21:p:7866-:d:951269. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.