IDEAS home Printed from https://ideas.repec.org/a/eee/renene/v252y2025ics096014812501198x.html

EOLD: A reinforcement learning-based energy-optimised load disaggregation framework for demand-side energy management

Author

Listed:
  • Wei, Ying'an
  • Fan, Jingjing
  • Meng, Qinglong
  • Debnath, Kumar Biswajit
  • Yang, Yuqin
  • Liu, Jiao
  • Lei, Yu

Abstract

Demand-Side energy Management (DSM) is a crucial strategy for balancing electricity supply and demand while enhancing energy efficiency, relying on sufficient data on electricity usage. Non-Intrusive Load Monitoring (NILM) is widely used for DSM strategies, as it effectively identifies the energy consumption of individual devices by measuring total power, significantly enhancing visibility. NILM should prioritise the dynamics of sub-load characteristics under future energy optimisation strategies rather than just historical data. For efficient load disaggregation, it must focus on optimising energy strategies. This study introduces a Reinforcement Learning-based Energy-Optimised Load Disaggregation (EOLD) framework to address this gap. The framework uses load disaggregation for final energy optimisation rather than initial sub-load characteristics. It utilises Reinforcement Learning (RL) to tackle the load disaggregation, with rewards focused on efficient, flexible, or economic energy goals. The Proximal Policy Optimisation (PPO) effectively disaggregates the air-conditioning load of three buildings, demonstrating the capabilities of the EOLD framework in optimising DSM for energy storage systems. The results show the proposed method optimises power curve flattening. It establishes a precise relationship between the main system's design power and the energy storage system's capacity. The framework can also be extended to disaggregate other flexible loads, such as photovoltaics and electric vehicles.

Suggested Citation

  • Wei, Ying'an & Fan, Jingjing & Meng, Qinglong & Debnath, Kumar Biswajit & Yang, Yuqin & Liu, Jiao & Lei, Yu, 2025. "EOLD: A reinforcement learning-based energy-optimised load disaggregation framework for demand-side energy management," Renewable Energy, Elsevier, vol. 252(C).
    • Handle: RePEc:eee:renene:v:252:y:2025:i:c:s096014812501198x
    DOI: 10.1016/j.renene.2025.123536
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S096014812501198X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.renene.2025.123536?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Liu, Bo & Luan, Wenpeng & Yu, Yixin, 2017. "Dynamic time warping based non-intrusive load transient identification," Applied Energy, Elsevier, vol. 195(C), pages 634-645.
    2. Kanakadhurga, Dharmaraj & Prabaharan, Natarajan, 2022. "Demand side management in microgrid: A critical review of key issues and recent trends," Renewable and Sustainable Energy Reviews, Elsevier, vol. 156(C).
    3. Zou, Wenke & Sun, Yongjun & Gao, Dian-ce & Zhang, Xu, 2023. "Globally optimal control of hybrid chilled water plants integrated with small-scale thermal energy storage for energy-efficient operation," Energy, Elsevier, vol. 262(PA).
    4. Rosales-Asensio, Enrique & Diez, David Borge & Sarmento, Paula, 2024. "Electricity balancing challenges for markets with high variable renewable generation," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PA).
    5. Kaneko, Naoya & Okazawa, Kazuki & Zhao, Dafang & Nishikawa, Hiroki & Taniguchi, Ittetsu & Murayama, Hiroyuki & Yura, Yoshinori & Okamoto, Masakazu & Catthoor, Francky & Onoye, Takao, 2024. "Non-intrusive thermal load disaggregation and forecasting for effective HVAC systems," Applied Energy, Elsevier, vol. 367(C).
    6. Lan, Penghang & Chen, She & Li, Qihang & Li, Kelin & Wang, Feng & Zhao, Yaoxun, 2024. "Intelligent hydrogen-ammonia combined energy storage system with deep reinforcement learning," Renewable Energy, Elsevier, vol. 237(PB).
    7. Yan, Lei & Tian, Wei & Han, Jiayu & Li, Zuy, 2022. "Event-driven two-stage solution to non-intrusive load monitoring," Applied Energy, Elsevier, vol. 311(C).
    8. Iqbal, Najam & Wang, Hu & Zheng, Zunqing & Yao, Mingfa, 2024. "Reinforcement learning-based heuristic planning for optimized energy management in power-split hybrid electric heavy duty vehicles," Energy, Elsevier, vol. 302(C).
    9. Bianchini, Gianni & Casini, Marco & Pepe, Daniele & Vicino, Antonio & Zanvettor, Giovanni Gino, 2019. "An integrated model predictive control approach for optimal HVAC and energy storage operation in large-scale buildings," Applied Energy, Elsevier, vol. 240(C), pages 327-340.
    10. Chen, Qi & Kuang, Zhonghong & Liu, Xiaohua & Zhang, Tao, 2024. "Application-oriented assessment of grid-connected PV-battery system with deep reinforcement learning in buildings considering electricity price dynamics," Applied Energy, Elsevier, vol. 364(C).
    11. Zhu, Ziqing & Hu, Ze & Chan, Ka Wing & Bu, Siqi & Zhou, Bin & Xia, Shiwei, 2023. "Reinforcement learning in deregulated energy market: A comprehensive review," Applied Energy, Elsevier, vol. 329(C).
    12. Liang, Tao & Chai, Lulu & Cao, Xin & Tan, Jianxin & Jing, Yanwei & Lv, Liangnian, 2024. "Real-time optimization of large-scale hydrogen production systems using off-grid renewable energy: Scheduling strategy based on deep reinforcement learning," Renewable Energy, Elsevier, vol. 224(C).
    13. Gao, Ang & Zheng, Jianyong & Mei, Fei & Liu, Yu, 2024. "Toward intelligent demand-side energy management via substation-level flexible load disaggregation," Applied Energy, Elsevier, vol. 367(C).
    14. Coraci, Davide & Brandi, Silvio & Hong, Tianzhen & Capozzoli, Alfonso, 2023. "Online transfer learning strategy for enhancing the scalability and deployment of deep reinforcement learning control in smart buildings," Applied Energy, Elsevier, vol. 333(C).
    15. Liang, Tao & Chai, Lulu & Tan, Jianxin & Jing, Yanwei & Lv, Liangnian, 2024. "Dynamic optimization of an integrated energy system with carbon capture and power-to-gas interconnection: A deep reinforcement learning-based scheduling strategy," Applied Energy, Elsevier, vol. 367(C).
    16. Mingzhe Zou & Shuyang Zhu & Jiacheng Gu & Lidija M. Korunovic & Sasa Z. Djokic, 2021. "Heating and Lighting Load Disaggregation Using Frequency Components and Convolutional Bidirectional Long Short-Term Memory Method," Energies, MDPI, vol. 14(16), pages 1-24, August.
    17. Patrick Huber & Alberto Calatroni & Andreas Rumsch & Andrew Paice, 2021. "Review on Deep Neural Networks Applied to Low-Frequency NILM," Energies, MDPI, vol. 14(9), pages 1-34, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Krzysztof Dowalla & Piotr Bilski & Robert Łukaszewski & Augustyn Wójcik & Ryszard Kowalik, 2022. "Application of the Time-Domain Signal Analysis for Electrical Appliances Identification in the Non-Intrusive Load Monitoring," Energies, MDPI, vol. 15(9), pages 1-20, May.
    2. Luo, Qingquan & Yu, Tao & Liang, Minhang & Pan, Zhenning & Guo, Wenlong & Hu, Xiaolei, 2025. "Review of advances in scaling non-intrusive load monitoring for real-world applications," Applied Energy, Elsevier, vol. 398(C).
    3. Li, Chuyi & Zheng, Kedi & Guo, Hongye & Chen, Qixin, 2023. "A mixed-integer programming approach for industrial non-intrusive load monitoring," Applied Energy, Elsevier, vol. 330(PA).
    4. Yan, Zhongzong & Nardello, Matteo & Brunelli, Davide & Wen, He, 2026. "Reproducible non-intrusive load monitoring: A survey of open-source methods and practical challenges," Renewable and Sustainable Energy Reviews, Elsevier, vol. 226(PA).
    5. Wang, Shiyu & Zheng, Yujing & Chen, Jing & Li, Zhaoxiang & Ji, Yuxiong & Du, Yuchuan, 2026. "Adaptive energy scheduling strategy for port logistics systems: A dual-consolidation continual reinforcement learning approach," Applied Energy, Elsevier, vol. 404(C).
    6. Razzano, Giuseppe & Brandi, Silvio & Piscitelli, Marco Savino & Capozzoli, Alfonso, 2025. "Rule extraction from deep reinforcement learning controller and comparative analysis with ASHRAE control sequences for the optimal management of Heating, Ventilation, and Air Conditioning (HVAC) systems in multizone buildings," Applied Energy, Elsevier, vol. 381(C).
    7. Emrani-Rahaghi, Pouria & Hashemi-Dezaki, Hamed & Ketabi, Abbas, 2023. "Efficient voltage control of low voltage distribution networks using integrated optimized energy management of networked residential multi-energy microgrids," Applied Energy, Elsevier, vol. 349(C).
    8. Félix González & Paul Arévalo & Luis Ramirez, 2025. "Game Theory and Robust Predictive Control for Peer-to-Peer Energy Management: A Pathway to a Low-Carbon Economy," Sustainability, MDPI, vol. 17(5), pages 1-23, February.
    9. Bianchini, Gianni & Casini, Marco & Gholami, Milad, 2025. "Optimal operation of renewable energy communities under demand response programs," Energy, Elsevier, vol. 326(C).
    10. İsmail Hakkı Çavdar & Vahit Feryad, 2021. "Efficient Design of Energy Disaggregation Model with BERT-NILM Trained by AdaX Optimization Method for Smart Grid," Energies, MDPI, vol. 14(15), pages 1-21, July.
    11. Yan, Lei & Tian, Wei & Wang, Hong & Hao, Xing & Li, Zuyi, 2023. "Robust event detection for residential load disaggregation," Applied Energy, Elsevier, vol. 331(C).
    12. Deng, Xu & Lv, Tao & Meng, Xiangyun & Li, Cong & Hou, Xiaoran & Xu, Jie & Wang, Yinhao & Liu, Feng, 2024. "Assessing the carbon emission reduction effect of flexibility option for integrating variable renewable energy," Energy Economics, Elsevier, vol. 132(C).
    13. Liu, Mingzhe & Guo, Mingyue & Fu, Yangyang & O’Neill, Zheng & Gao, Yuan, 2024. "Expert-guided imitation learning for energy management: Evaluating GAIL’s performance in building control applications," Applied Energy, Elsevier, vol. 372(C).
    14. Zhao, Yincheng & Zhang, Guozhou & Hu, Weihao & Huang, Qi & Chen, Zhe & Blaabjerg, Frede, 2023. "Meta-learning based voltage control strategy for emergency faults of active distribution networks," Applied Energy, Elsevier, vol. 349(C).
    15. Shi, Xin & Ming, Hao & Shakkottai, Srinivas & Xie, Le & Yao, Jianguo, 2019. "Nonintrusive load monitoring in residential households with low-resolution data," Applied Energy, Elsevier, vol. 252(C), pages 1-1.
    16. Lan, Penghang & Chen, She & Li, Qihang & Li, Kelin & Wang, Feng & Zhao, Yaoxun, 2024. "Intelligent hydrogen-ammonia combined energy storage system with deep reinforcement learning," Renewable Energy, Elsevier, vol. 237(PB).
    17. Liu, Hong & Zhao, Yue & Gu, Chenghong & Ge, Shaoyun & Yang, Zan, 2021. "Adjustable capability of the distributed energy system: Definition, framework, and evaluation model," Energy, Elsevier, vol. 222(C).
    18. Liu, Bo & Hou, Yufan & Luan, Wenpeng & Liu, Zishuai & Chen, Sheng & Yu, Yixin, 2023. "A divide-and-conquer method for compression and reconstruction of smart meter data," Applied Energy, Elsevier, vol. 336(C).
    19. Feng, Yiwei & Li, Yanpeng & Qu, Shengli & Liu, Yishuang & Wang, Chuang & Han, Yaoxiang & Xing, Ziwen, 2025. "Proactive operational strategy of thermal energy storage tank in an industrial multi-chiller system based on chilled water flow difference between supply and demand sides," Energy, Elsevier, vol. 319(C).
    20. Zhang, Liwen & Liu, TingHao & Xu, Chenzhou & Zhang, Jun & Gao, Yujie & Li, Xin Cheng & Wang, HaoTian & Li, Xin & Mir, Mahdi, 2025. "A risk-averse cooperative framework for neighboring energy hubs under joint carbon, heat and electricity trading market with P2G and renewables," Renewable Energy, Elsevier, vol. 250(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:renene:v:252:y:2025:i:c:s096014812501198x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/renewable-energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.