IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v356y2024ics030626192301721x.html
   My bibliography  Save this article

Deep clustering of reinforcement learning based on the bang-bang principle to optimize the energy in multi-boiler for intelligent buildings

Author

Listed:
  • Homod, Raad Z.
  • Munahi, Basil Sh.
  • Mohammed, Hayder Ibrahim
  • Albadr, Musatafa Abbas Abbood
  • Abderrahmane, AISSA
  • Mahdi, Jasim M.
  • Ben Hamida, Mohamed Bechir
  • Alhasnawi, Bilal Naji
  • Albahri, A.S.
  • Togun, Hussein
  • Alqsair, Umar F.
  • Yaseen, Zaher Mundher

Abstract

The bang-bang relays of the multiple-boiler system (MBS) control, are characterized by complex limiter saturation functions and classified as fixed parameters. Their action signals cannot precisely control the nonlinear dynamic building heating demand over their entire range of operation. Moreover, in a mono-boiler system, the bang-bang controller endures increasing short cycling over partial load time due to the heating system being considered to have an oversized boiler at most times of running, thus promoting high energy consumption and fluctuating indoor thermal comfort. So, it is difficult to cope with uncertainties in outdoor environments and indoor heating load. Hence, this study formulates the MBS control problem as a dynamic Markov decision process and applies a deep clustering of reinforcement learning approach to obtain the optimal control policy through interaction with the environment based on multi-agent learning according to bang-bang action. With such an approach, adopting a new boiler sequencing control (BSC) strategy using deep clustering of reinforcement learning based on a bang-bang (DCRLBB) manner. The deep clustering is configured to break Lagrangian trajectory curves into piecewise segments to represent the RL agent's action policy. The agent's action policy signals are configured from the bang-bang reward formula based on trade-off implications to be more adjustable than traditional fixed parameters such as fuzzy bang-bang controller (FBBC). The agent of BSC significantly affects the energy performance of the MBS, whereas the other agent resizes boiler capacity by acting to adjust the boiler solenoid fuel valve. The comparison of results between the proposed strategy and conventional FBBC shows distinct differences in the superior response of DCRLBB under dynamic indoor/outdoor actual conditions and energy saving by more than 32% while maintaining the indoor thermal in the comfortable range.

Suggested Citation

  • Homod, Raad Z. & Munahi, Basil Sh. & Mohammed, Hayder Ibrahim & Albadr, Musatafa Abbas Abbood & Abderrahmane, AISSA & Mahdi, Jasim M. & Ben Hamida, Mohamed Bechir & Alhasnawi, Bilal Naji & Albahri, A., 2024. "Deep clustering of reinforcement learning based on the bang-bang principle to optimize the energy in multi-boiler for intelligent buildings," Applied Energy, Elsevier, vol. 356(C).
    • Handle: RePEc:eee:appene:v:356:y:2024:i:c:s030626192301721x
    DOI: 10.1016/j.apenergy.2023.122357
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S030626192301721X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2023.122357?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Homod, Raad Z., 2014. "Assessment regarding energy saving and decoupling for different AHU (air handling unit) and control strategies in the hot-humid climatic region of Iraq," Energy, Elsevier, vol. 74(C), pages 762-774.
    2. Homod, Raad Z. & Mohammed, Hayder Ibrahim & Abderrahmane, Aissa & Alawi, Omer A. & Khalaf, Osamah Ibrahim & Mahdi, Jasim M. & Guedri, Kamel & Dhaidan, Nabeel S. & Albahri, A.S. & Sadeq, Abdellatif M. , 2023. "Deep clustering of Lagrangian trajectory for multi-task learning to energy saving in intelligent buildings using cooperative multi-agent," Applied Energy, Elsevier, vol. 351(C).
    3. Fujishima, Hidekatsu & Takekoshi, Kenichi & Kuroki, Tomoyuki & Tanaka, Atsushi & Otsuka, Keiichi & Okubo, Masaaki, 2013. "Towards ideal NOx control technology for bio-oils and a gas multi-fuel boiler system using a plasma-chemical hybrid process," Applied Energy, Elsevier, vol. 111(C), pages 394-400.
    4. Maytham S. Ahmed & Azah Mohamed & Raad Z. Homod & Hussain Shareef, 2016. "Hybrid LSA-ANN Based Home Energy Management Scheduling Controller for Residential Demand Response Strategy," Energies, MDPI, vol. 9(9), pages 1-20, September.
    5. Wang, Kang & Wang, Haixin & Yang, Zihao & Feng, Jiawei & Li, Yanzhen & Yang, Junyou & Chen, Zhe, 2023. "A transfer learning method for electric vehicles charging strategy based on deep reinforcement learning," Applied Energy, Elsevier, vol. 343(C).
    6. Homod, Raad Z. & Gaeid, Khalaf S. & Dawood, Suroor M. & Hatami, Alireza & Sahari, Khairul S., 2020. "Evaluation of energy-saving potential for optimal time response of HVAC control system in smart buildings," Applied Energy, Elsevier, vol. 271(C).
    7. Homod, Raad Z. & Sahari, Khairul Salleh Mohamed & Almurib, Haider A.F., 2014. "Energy saving by integrated control of natural ventilation and HVAC systems using model guide for comparison," Renewable Energy, Elsevier, vol. 71(C), pages 639-650.
    8. Liao, Weicheng & Zhang, Xiaoyue & Li, Zhen, 2022. "Experimental investigation on the performance of a boiler system with flue gas dehumidification and combustion air humidification," Applied Energy, Elsevier, vol. 323(C).
    9. Homod, Raad Z. & Togun, Hussein & Kadhim Hussein, Ahmed & Noraldeen Al-Mousawi, Fadhel & Yaseen, Zaher Mundher & Al-Kouz, Wael & Abd, Haider J. & Alawi, Omer A. & Goodarzi, Marjan & Hussein, Omar A., 2022. "Dynamics analysis of a novel hybrid deep clustering for unsupervised learning by reinforcement of multi-agent to energy saving in intelligent buildings," Applied Energy, Elsevier, vol. 313(C).
    10. Homod, Raad Z. & Togun, Hussein & Ateeq, Adnan A. & Al-Mousawi, Fadhel Noraldeen & Yaseen, Zaher Mundher & Al-Kouz, Wael & Hussein, Ahmed Kadhim & Alawi, Omer A. & Goodarzi, Marjan & Ahmadi, Goodarz, 2022. "An innovative clustering technique to generate hybrid modeling of cooling coils for energy analysis: A case study for control performance in HVAC systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 166(C).
    11. Zheng, Peijun & Zhou, Heng & Liu, Jiang & Nakanishi, Yosuke, 2023. "Interpretable building energy consumption forecasting using spectral clustering algorithm and temporal fusion transformers architecture," Applied Energy, Elsevier, vol. 349(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Homod, Raad Z. & Mohammed, Hayder Ibrahim & Abderrahmane, Aissa & Alawi, Omer A. & Khalaf, Osamah Ibrahim & Mahdi, Jasim M. & Guedri, Kamel & Dhaidan, Nabeel S. & Albahri, A.S. & Sadeq, Abdellatif M. , 2023. "Deep clustering of Lagrangian trajectory for multi-task learning to energy saving in intelligent buildings using cooperative multi-agent," Applied Energy, Elsevier, vol. 351(C).
    2. Homod, Raad Z. & Togun, Hussein & Ateeq, Adnan A. & Al-Mousawi, Fadhel Noraldeen & Yaseen, Zaher Mundher & Al-Kouz, Wael & Hussein, Ahmed Kadhim & Alawi, Omer A. & Goodarzi, Marjan & Ahmadi, Goodarz, 2022. "An innovative clustering technique to generate hybrid modeling of cooling coils for energy analysis: A case study for control performance in HVAC systems," Renewable and Sustainable Energy Reviews, Elsevier, vol. 166(C).
    3. Homod, Raad Z. & Togun, Hussein & Kadhim Hussein, Ahmed & Noraldeen Al-Mousawi, Fadhel & Yaseen, Zaher Mundher & Al-Kouz, Wael & Abd, Haider J. & Alawi, Omer A. & Goodarzi, Marjan & Hussein, Omar A., 2022. "Dynamics analysis of a novel hybrid deep clustering for unsupervised learning by reinforcement of multi-agent to energy saving in intelligent buildings," Applied Energy, Elsevier, vol. 313(C).
    4. Homod, Raad Z., 2018. "Analysis and optimization of HVAC control systems based on energy and performance considerations for smart buildings," Renewable Energy, Elsevier, vol. 126(C), pages 49-64.
    5. Homod, Raad Z. & Gaeid, Khalaf S. & Dawood, Suroor M. & Hatami, Alireza & Sahari, Khairul S., 2020. "Evaluation of energy-saving potential for optimal time response of HVAC control system in smart buildings," Applied Energy, Elsevier, vol. 271(C).
    6. Alexandre Correia & Luís Miguel Ferreira & Paulo Coimbra & Pedro Moura & Aníbal T. de Almeida, 2022. "Smart Thermostats for a Campus Microgrid: Demand Control and Improving Air Quality," Energies, MDPI, vol. 15(4), pages 1-21, February.
    7. Waseem, Muhammad & Lin, Zhenzhi & Liu, Shengyuan & Zhang, Zhi & Aziz, Tarique & Khan, Danish, 2021. "Fuzzy compromised solution-based novel home appliances scheduling and demand response with optimal dispatch of distributed energy resources," Applied Energy, Elsevier, vol. 290(C).
    8. Lei, Yue & Zhan, Sicheng & Ono, Eikichi & Peng, Yuzhen & Zhang, Zhiang & Hasama, Takamasa & Chong, Adrian, 2022. "A practical deep reinforcement learning framework for multivariate occupant-centric control in buildings," Applied Energy, Elsevier, vol. 324(C).
    9. Kritana Prueksakorn & Cheng-Xu Piao & Hyunchul Ha & Taehyeung Kim, 2015. "Computational and Experimental Investigation for an Optimal Design of Industrial Windows to Allow Natural Ventilation during Wind-Driven Rain," Sustainability, MDPI, vol. 7(8), pages 1-22, August.
    10. Adnan Ahmad & Asif Khan & Nadeem Javaid & Hafiz Majid Hussain & Wadood Abdul & Ahmad Almogren & Atif Alamri & Iftikhar Azim Niaz, 2017. "An Optimized Home Energy Management System with Integrated Renewable Energy and Storage Resources," Energies, MDPI, vol. 10(4), pages 1-35, April.
    11. Lee, Minjung & Ham, Jeonggyun & Lee, Jeong-Won & Cho, Honghyun, 2023. "Analysis of thermal comfort, energy consumption, and CO2 reduction of indoor space according to the type of local heating under winter rest conditions," Energy, Elsevier, vol. 268(C).
    12. Khan, Zulfiqar Ahmad & Khan, Shabbir Ahmad & Hussain, Tanveer & Baik, Sung Wook, 2024. "DSPM: Dual sequence prediction model for efficient energy management in micro-grid," Applied Energy, Elsevier, vol. 356(C).
    13. Lei Chen & Hongkun Chen & Jun Yang & Yanjuan Yu & Kaiwei Zhen & Yang Liu & Li Ren, 2017. "Coordinated Control of Superconducting Fault Current Limiter and Superconducting Magnetic Energy Storage for Transient Performance Enhancement of Grid-Connected Photovoltaic Generation System," Energies, MDPI, vol. 10(1), pages 1-23, January.
    14. Salah L. Zubaidi & Sadik K. Gharghan & Jayne Dooley & Rafid M. Alkhaddar & Mawada Abdellatif, 2018. "Short-Term Urban Water Demand Prediction Considering Weather Factors," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 32(14), pages 4527-4542, November.
    15. Amit Shewale & Anil Mokhade & Nitesh Funde & Neeraj Dhanraj Bokde, 2022. "A Survey of Efficient Demand-Side Management Techniques for the Residential Appliance Scheduling Problem in Smart Homes," Energies, MDPI, vol. 15(8), pages 1-34, April.
    16. Abdul Mujeebu, Muhammad & Alshamrani, Othman Subhi, 2016. "Prospects of energy conservation and management in buildings – The Saudi Arabian scenario versus global trends," Renewable and Sustainable Energy Reviews, Elsevier, vol. 58(C), pages 1647-1663.
    17. Chenari, Behrang & Dias Carrilho, João & Gameiro da Silva, Manuel, 2016. "Towards sustainable, energy-efficient and healthy ventilation strategies in buildings: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 59(C), pages 1426-1447.
    18. Chen, Yujiao & Tong, Zheming & Wu, Wentao & Samuelson, Holly & Malkawi, Ali & Norford, Leslie, 2019. "Achieving natural ventilation potential in practice: Control schemes and levels of automation," Applied Energy, Elsevier, vol. 235(C), pages 1141-1152.
    19. Nizami, Sohrab & Tushar, Wayes & Hossain, M.J. & Yuen, Chau & Saha, Tapan & Poor, H. Vincent, 2022. "Transactive energy for low voltage residential networks: A review," Applied Energy, Elsevier, vol. 323(C).
    20. Krzysztof Lalik & Mateusz Kozek & Szymon Podlasek & Rafał Figaj & Paweł Gut, 2021. "Q-Learning Neural Controller for Steam Generator Station in Micro Cogeneration Systems," Energies, MDPI, vol. 14(17), pages 1-13, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:356:y:2024:i:c:s030626192301721x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.