IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v355y2024ics0306261923016227.html
   My bibliography  Save this article

A data-driven DRL-based home energy management system optimization framework considering uncertain household parameters

Author

Listed:
  • Ren, Kezheng
  • Liu, Jun
  • Wu, Zeyang
  • Liu, Xinglei
  • Nie, Yongxin
  • Xu, Haitao

Abstract

With the rise in household computing power and the increasing number of smart devices, more and more residents are able to participate in demand response (DR) management through the home energy management system (HEMS). However, HEMS has encountered challenges in developing the most effective energy management strategies, including the complexity of modeling user comfort, uncertainty in electricity price and photovoltaic (PV) output, and the challenge of solving high-dimensional time-coupled decision problems. To address these challenges, a novel data-driven deep reinforcement learning (DRL)-base HEMS optimization framework considering uncertain household parameters is proposed. Firstly, a thermal comfort evaluation model based on integrated learning is proposed. Then, a prediction model based on the bidirectional gated recurrent unit neural network (BiGRU-NN) algorithm is proposed to mine the time series PV output and electricity price data. Finally, combining the PV output and electricity price forecasting, along with the thermal comfort evaluation, an optimal decision-making method based on soft actor-critic (SAC) algorithm for the HEMS is established. The results of numerical experiments show that the proposed method can effectively solve the high-dimensional integrated decision-making problem with uncertainty. By participating in DR, the household electricity cost can be reduced by 17.7% and the total cost can be reduced by 8.4%. Furthermore, the comparison result shows that the method proposed in this paper performs better than the existing optimization models based on proximal policy optimization (PPO) algorithm and twin-delayed depth deterministic policy gradient (TD3) algorithm.

Suggested Citation

  • Ren, Kezheng & Liu, Jun & Wu, Zeyang & Liu, Xinglei & Nie, Yongxin & Xu, Haitao, 2024. "A data-driven DRL-based home energy management system optimization framework considering uncertain household parameters," Applied Energy, Elsevier, vol. 355(C).
    • Handle: RePEc:eee:appene:v:355:y:2024:i:c:s0306261923016227
    DOI: 10.1016/j.apenergy.2023.122258
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261923016227
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2023.122258?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Zhou, Yuekuan & Zheng, Siqian & Zhang, Guoqiang, 2020. "Machine-learning based study on the on-site renewable electrical performance of an optimal hybrid PCMs integrated renewable system with high-level parameters’ uncertainties," Renewable Energy, Elsevier, vol. 151(C), pages 403-418.
    2. Dinh, Huy Truong & Lee, Kyu-haeng & Kim, Daehee, 2022. "Supervised-learning-based hour-ahead demand response for a behavior-based home energy management system approximating MILP optimization," Applied Energy, Elsevier, vol. 321(C).
    3. Zhou, Yuekuan & Zheng, Siqian, 2020. "Machine-learning based hybrid demand-side controller for high-rise office buildings with high energy flexibilities," Applied Energy, Elsevier, vol. 262(C).
    4. Zhou, Yuekuan, 2022. "Demand response flexibility with synergies on passive PCM walls, BIPVs, and active air-conditioning system in a subtropical climate," Renewable Energy, Elsevier, vol. 199(C), pages 204-225.
    5. Nakıp, Mert & Çopur, Onur & Biyik, Emrah & Güzeliş, Cüneyt, 2023. "Renewable energy management in smart home environment via forecast embedded scheduling based on Recurrent Trend Predictive Neural Network," Applied Energy, Elsevier, vol. 340(C).
    6. Gao, Yuan & Matsunami, Yuki & Miyata, Shohei & Akashi, Yasunori, 2022. "Operational optimization for off-grid renewable building energy system using deep reinforcement learning," Applied Energy, Elsevier, vol. 325(C).
    7. Tostado-Véliz, Marcos & Kamel, Salah & Aymen, Flah & Jurado, Francisco, 2022. "A novel hybrid lexicographic-IGDT methodology for robust multi-objective solution of home energy management systems," Energy, Elsevier, vol. 253(C).
    8. Lu, Renzhi & Bai, Ruichang & Ding, Yuemin & Wei, Min & Jiang, Junhui & Sun, Mingyang & Xiao, Feng & Zhang, Hai-Tao, 2021. "A hybrid deep learning-based online energy management scheme for industrial microgrid," Applied Energy, Elsevier, vol. 304(C).
    9. Zhou, Yuekuan & Zheng, Siqian, 2020. "Multi-level uncertainty optimisation on phase change materials integrated renewable systems with hybrid ventilations and active cooling," Energy, Elsevier, vol. 202(C).
    10. Liu, Xinglei & Liu, Jun & Ren, Kezheng & Liu, Xiaoming & Liu, Jiacheng, 2022. "An integrated fuzzy multi-energy transaction evaluation approach for energy internet markets considering judgement credibility and variable rough precision," Energy, Elsevier, vol. 261(PB).
    11. Zhang, Chu & Ma, Huixin & Hua, Lei & Sun, Wei & Nazir, Muhammad Shahzad & Peng, Tian, 2022. "An evolutionary deep learning model based on TVFEMD, improved sine cosine algorithm, CNN and BiLSTM for wind speed prediction," Energy, Elsevier, vol. 254(PA).
    12. Chreim, Bashar & Esseghir, Moez & Merghem-Boulahia, Leila, 2022. "LOSISH—LOad Scheduling In Smart Homes based on demand response: Application to smart grids," Applied Energy, Elsevier, vol. 323(C).
    13. Chen, Zhang & Liu, Jun & Liu, Xinglei, 2022. "GPU accelerated power flow calculation of integrated electricity and heat system with component-oriented modeling of district heating network," Applied Energy, Elsevier, vol. 305(C).
    14. Ren, Kezheng & Liu, Jun & Liu, Xinglei & Nie, Yongxin, 2023. "Reinforcement Learning-Based Bi-Level strategic bidding model of Gas-fired unit in integrated electricity and natural gas markets preventing market manipulation," Applied Energy, Elsevier, vol. 336(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Qin, Di & Liu, Zhengxuan & Zhou, Yuekuan & Yan, Zhongjun & Chen, Dachuan & Zhang, Guoqiang, 2021. "Dynamic performance of a novel air-soil heat exchanger coupling with diversified energy storage components—modelling development, experimental verification, parametrical design and robust operation," Renewable Energy, Elsevier, vol. 167(C), pages 542-557.
    2. Zhou, Yuekuan & Zheng, Siqian & Liu, Zhengxuan & Wen, Tao & Ding, Zhixiong & Yan, Jun & Zhang, Guoqiang, 2020. "Passive and active phase change materials integrated building energy systems with advanced machine-learning based climate-adaptive designs, intelligent operations, uncertainty-based analysis and optim," Renewable and Sustainable Energy Reviews, Elsevier, vol. 130(C).
    3. Zhou, Yuekuan & Zheng, Siqian, 2020. "Uncertainty study on thermal and energy performances of a deterministic parameters based optimal aerogel glazing system using machine-learning method," Energy, Elsevier, vol. 193(C).
    4. Nakıp, Mert & Çopur, Onur & Biyik, Emrah & Güzeliş, Cüneyt, 2023. "Renewable energy management in smart home environment via forecast embedded scheduling based on Recurrent Trend Predictive Neural Network," Applied Energy, Elsevier, vol. 340(C).
    5. Gohar Gholamibozanjani & Mohammed Farid, 2021. "A Critical Review on the Control Strategies Applied to PCM-Enhanced Buildings," Energies, MDPI, vol. 14(7), pages 1-39, March.
    6. Liu, Zhengxuan & Sun, Pengchen & Xie, Mingjing & Zhou, Yuekuan & He, Yingdong & Zhang, Guoqiang & Chen, Dachuan & Li, Shuisheng & Yan, Zhongjun & Qin, Di, 2021. "Multivariant optimization and sensitivity analysis of an experimental vertical earth-to-air heat exchanger system integrating phase change material with Taguchi method," Renewable Energy, Elsevier, vol. 173(C), pages 401-414.
    7. Zhou, Yuekuan, 2022. "Demand response flexibility with synergies on passive PCM walls, BIPVs, and active air-conditioning system in a subtropical climate," Renewable Energy, Elsevier, vol. 199(C), pages 204-225.
    8. He, Zhaoyu & Guo, Weimin & Zhang, Peng, 2022. "Performance prediction, optimal design and operational control of thermal energy storage using artificial intelligence methods," Renewable and Sustainable Energy Reviews, Elsevier, vol. 156(C).
    9. Zhou, Yuekuan & Zheng, Siqian, 2020. "Climate adaptive optimal design of an aerogel glazing system with the integration of a heuristic teaching-learning-based algorithm in machine learning-based optimization," Renewable Energy, Elsevier, vol. 153(C), pages 375-391.
    10. Emrani-Rahaghi, Pouria & Hashemi-Dezaki, Hamed & Ketabi, Abbas, 2023. "Efficient voltage control of low voltage distribution networks using integrated optimized energy management of networked residential multi-energy microgrids," Applied Energy, Elsevier, vol. 349(C).
    11. Lu, Yu & Xiang, Yue & Huang, Yuan & Yu, Bin & Weng, Liguo & Liu, Junyong, 2023. "Deep reinforcement learning based optimal scheduling of active distribution system considering distributed generation, energy storage and flexible load," Energy, Elsevier, vol. 271(C).
    12. Zhou, Yuekuan & Zheng, Siqian, 2020. "Multi-level uncertainty optimisation on phase change materials integrated renewable systems with hybrid ventilations and active cooling," Energy, Elsevier, vol. 202(C).
    13. Meng, Anbo & Zhang, Haitao & Yin, Hao & Xian, Zikang & Chen, Shu & Zhu, Zibin & Zhang, Zheng & Rong, Jiayu & Li, Chen & Wang, Chenen & Wu, Zhenbo & Deng, Weisi & Luo, Jianqiang & Wang, Xiaolin, 2023. "A novel multi-gradient evolutionary deep learning approach for few-shot wind power prediction using time-series GAN," Energy, Elsevier, vol. 283(C).
    14. Keerthana Sivamayil & Elakkiya Rajasekar & Belqasem Aljafari & Srete Nikolovski & Subramaniyaswamy Vairavasundaram & Indragandhi Vairavasundaram, 2023. "A Systematic Study on Reinforcement Learning Based Applications," Energies, MDPI, vol. 16(3), pages 1-23, February.
    15. Deng, Xiangtian & Zhang, Yi & Jiang, Yi & Zhang, Yi & Qi, He, 2024. "A novel operation method for renewable building by combining distributed DC energy system and deep reinforcement learning," Applied Energy, Elsevier, vol. 353(PB).
    16. Elkholy, M.H. & Metwally, Hamid & Farahat, M.A. & Senjyu, Tomonobu & Elsayed Lotfy, Mohammed, 2022. "Smart centralized energy management system for autonomous microgrid using FPGA," Applied Energy, Elsevier, vol. 317(C).
    17. Yuan, Yu & Bai, Zhang & Zhou, Shengdong & Zheng, Bo & Hu, Wenxin, 2022. "Potential of applying the thermochemical recuperation in combined cooling, heating and power generation: Flexible demand response characteristics," Applied Energy, Elsevier, vol. 325(C).
    18. Lv, Sheng-Xiang & Wang, Lin, 2023. "Multivariate wind speed forecasting based on multi-objective feature selection approach and hybrid deep learning model," Energy, Elsevier, vol. 263(PE).
    19. Nebiyu Kedir & Phuong H. D. Nguyen & Citlaly Pérez & Pedro Ponce & Aminah Robinson Fayek, 2023. "Systematic Literature Review on Fuzzy Hybrid Methods in Photovoltaic Solar Energy: Opportunities, Challenges, and Guidance for Implementation," Energies, MDPI, vol. 16(9), pages 1-38, April.
    20. Wang, Sen & Qin, Chaoxu & Feng, Qihong & Javadpour, Farzam & Rui, Zhenhua, 2021. "A framework for predicting the production performance of unconventional resources using deep learning," Applied Energy, Elsevier, vol. 295(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:355:y:2024:i:c:s0306261923016227. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.