IDEAS home Printed from https://ideas.repec.org/a/eee/appene/v402y2025ipas0306261925015934.html

Deep reinforcement learning for HVAC control with nonlinear parametric thermal network modeling for passive building envelopes

Author

Listed:
  • Lu, Yu
  • Wang, Wenqi
  • Wang, Chuyao
  • Li, Ze
  • Zhou, Yiying
  • Chen, Xu
  • Ho, Tsz Chung
  • Tso, Chi Yan

Abstract

The incorporation of passive cooling envelopes into buildings can effectively reduce energy consumption. However, due to their limited cooling capacity, HVAC systems are still required to maintain indoor thermal comfort. In buildings using passive radiative cooling roofs and thermochromic windows as passive cooling envelopes, inappropriate HVAC control strategies are more likely to occur due to the changeable optical and thermal properties. Such improper HVAC control results in significant waste during system operation, which remains an unsolved problem. Therefore, optimizing HVAC operation in passively cooled buildings is essential not only for ensuring thermal comfort but also for further reducing energy consumption. Achieving both objectives depends on effectively capturing the building's thermal behavior and efficient control methods. To incorporate the thermal behavior of passive cooling envelopes into the HVAC control system, this study first develops a resistance-capacitance thermal network based on a modified matrix to predict the thermal behavior of passively cooled buildings. Then, a model-based policy optimization deep reinforcement learning (DRL) control method is proposed to enhance HVAC system performance in such buildings. The results show that the modified matrix significantly improves the prediction accuracy of the thermal behavior of passively cooled buildings compared to current global identification methods, which increases coefficient of determination from 0.90, 0.73, 0.88 to 0.94, 0.92, 0.95 for radiative cooling roofs, thermochromic windows, and a combination of both envelopes, respectively. Moreover, the proposed DRL control method can reduce building energy consumption by 17.7 % for radiative cooling roofs, 10.6 % for thermochromic windows, and 21.1 % when both strategies are applied simultaneously, compared to the baseline control method. This study provides valuable insights into optimization of HVAC system operations in buildings equipped with passive cooling envelopes.

Suggested Citation

  • Lu, Yu & Wang, Wenqi & Wang, Chuyao & Li, Ze & Zhou, Yiying & Chen, Xu & Ho, Tsz Chung & Tso, Chi Yan, 2025. "Deep reinforcement learning for HVAC control with nonlinear parametric thermal network modeling for passive building envelopes," Applied Energy, Elsevier, vol. 402(PA).
    • Handle: RePEc:eee:appene:v:402:y:2025:i:pa:s0306261925015934
    DOI: 10.1016/j.apenergy.2025.126863
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0306261925015934
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.apenergy.2025.126863?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Wang, Ze-Ye & Wu, Xian & Qu, Ming-Liang & Fan, Li-Wu & Yu, Zi-Tao & Chen, Shu-Qin & Ge, Jian & Wang, Liang & Dai, Sheng-Juan, 2025. "A field test and evaluation of radiative cooling performance as applied on the sidewall surfaces of residential buildings in China," Applied Energy, Elsevier, vol. 379(C).
    2. Jianing Song & Wenluan Zhang & Zhengnan Sun & Mengyao Pan & Feng Tian & Xiuhong Li & Ming Ye & Xu Deng, 2022. "Durable radiative cooling against environmental aging," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    3. Yang, Shiyu & Wan, Man Pun, 2022. "Machine-learning-based model predictive control with instantaneous linearization – A case study on an air-conditioning and mechanical ventilation system," Applied Energy, Elsevier, vol. 306(PB).
    4. Vázquez-Canteli, José R. & Nagy, Zoltán, 2019. "Reinforcement learning for demand response: A review of algorithms and modeling techniques," Applied Energy, Elsevier, vol. 235(C), pages 1072-1089.
    5. Zhang, Yi & Tennakoon, Thilhara & Chan, Yin Hoi & Chan, Ka Chung & Fu, Sau Chung & Tso, Chi Yan & Yu, Kin Man & Huang, Bao Ling & Yao, Shu Huai & Qiu, Hui He & Chao, Christopher Y.H., 2022. "Energy consumption modelling of a passive hybrid system for office buildings in different climates," Energy, Elsevier, vol. 239(PA).
    6. Giovannini, Luigi & Favoino, Fabio & Pellegrino, Anna & Lo Verso, Valerio Roberto Maria & Serra, Valentina & Zinzi, Michele, 2019. "Thermochromic glazing performance: From component experimental characterisation to whole building performance evaluation," Applied Energy, Elsevier, vol. 251(C), pages 1-1.
    7. Bu, Fan & Yan, Da & Tan, Gang & Sun, Hongsan & An, Jingjing, 2022. "Systematically incorporating spectrum-selective radiative cooling into building performance simulation: Numerical integration method and experimental validation," Applied Energy, Elsevier, vol. 312(C).
    8. Huaiyuan Wang & Yuanwei Lu & Jie Wang & Tao Qi & Xuefeng Tian & Chaowei Yang & Yuming Huang & Meiqi Wang & Baiqi Zhang & Zhibin Qu & Wei Zhou & Fei Sun & Jihui Gao & Guangbo Zhao, 2025. "Hydrated ionic polymer for thermochromic smart windows in buildings," Nature Communications, Nature, vol. 16(1), pages 1-11, December.
    9. Wang, Hao & Chen, Xiwen & Vital, Natan & Duffy, Edward & Razi, Abolfazl, 2024. "Energy optimization for HVAC systems in multi-VAV open offices: A deep reinforcement learning approach," Applied Energy, Elsevier, vol. 356(C).
    10. Aaswath P. Raman & Marc Abou Anoma & Linxiao Zhu & Eden Rephaeli & Shanhui Fan, 2014. "Passive radiative cooling below ambient air temperature under direct sunlight," Nature, Nature, vol. 515(7528), pages 540-544, November.
    11. Bu, Fan & Yan, Da & Tan, Gang & An, Jingjing, 2024. "A novel approach based on equivalent sky radiative temperature for quick computation of radiative cooling in building energy simulation," Renewable Energy, Elsevier, vol. 221(C).
    12. Zhang, Wenshuo & Jiao, Dongsheng & Zhao, Bin & Pei, Gang, 2024. "Experimental and numerical investigation of the effects of passive radiative cooling-based cool roof on building energy consumption," Applied Energy, Elsevier, vol. 376(PA).
    13. Silvestri, Alberto & Coraci, Davide & Brandi, Silvio & Capozzoli, Alfonso & Borkowski, Esther & Köhler, Johannes & Wu, Duan & Zeilinger, Melanie N. & Schlueter, Arno, 2024. "Real building implementation of a deep reinforcement learning controller to enhance energy efficiency and indoor temperature control," Applied Energy, Elsevier, vol. 368(C).
    14. Klingebiel, Jonas & Höges, Christoph & Horst, Janik & Nießen, Oliver & Venzik, Valerius & Vering, Christian & Müller, Dirk, 2025. "A self-optimizing defrost initiation controller for air-source heat pumps: Experimental validation of deep reinforcement learning," Applied Energy, Elsevier, vol. 398(C).
    15. C. Sanama & X. Xia & M. Nguepnang & Sheng Du, 2022. "PID-MPC Implementation on a Chiller-Fan Coil Unit," Journal of Mathematics, Hindawi, vol. 2022, pages 1-13, September.
    16. Zhen Chen & Linxiao Zhu & Aaswath Raman & Shanhui Fan, 2016. "Radiative cooling to deep sub-freezing temperatures through a 24-h day–night cycle," Nature Communications, Nature, vol. 7(1), pages 1-5, December.
    17. Du, Yan & Zandi, Helia & Kotevska, Olivera & Kurte, Kuldeep & Munk, Jeffery & Amasyali, Kadir & Mckee, Evan & Li, Fangxing, 2021. "Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning," Applied Energy, Elsevier, vol. 281(C).
    18. Chen, Jianheng & Lu, Lin & Gong, Quan, 2023. "Techno-economic and environmental evaluation on radiative sky cooling-based novel passive envelope strategies to achieve building sustainability and carbon neutrality," Applied Energy, Elsevier, vol. 349(C).
    19. Chao Ding & Jing Ke & Mark Levine & Jessica Granderson & Nan Zhou, 2024. "Potential of artificial intelligence in reducing energy and carbon emissions of commercial buildings at scale," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    20. Wu, Shuangdui & Sun, Hongli & Song, Junkang & Liu, Sai & Shi, Shaohang & Tso, ChiYan & Lin, Borong, 2024. "Comprehensive analysis on building performance enhancement based on selective split-band modulated adaptive thermochromic windows," Applied Energy, Elsevier, vol. 372(C).
    21. Sha, Xinyi & Ma, Zhenjun & Sethuvenkatraman, Subbu & Li, Wanqing, 2025. "Online learning-enhanced data-driven model predictive control for optimizing HVAC energy consumption, indoor air quality and thermal comfort," Applied Energy, Elsevier, vol. 383(C).
    22. Chen, Minghao & Xie, Zhiyuan & Sun, Yi & Zheng, Shunlin, 2023. "The predictive management in campus heating system based on deep reinforcement learning and probabilistic heat demands forecasting," Applied Energy, Elsevier, vol. 350(C).
    23. Liu, Qiong & Guo, Ye & Xu, Tong, 2025. "Robust deep reinforcement learning for inverter-based volt-var control in partially observable distribution networks," Applied Energy, Elsevier, vol. 399(C).
    24. Wang, Chuyao & Yang, Hongxing & Ji, Jie, 2024. "Performance analysis of a PV/T shading device for enhancing energy saving and human comfort," Applied Energy, Elsevier, vol. 376(PA).
    25. Liao, Chenxin & Miyata, Shohei & Qu, Ming & Akashi, Yasunori, 2025. "Year-round operational optimization of HVAC systems using hierarchical deep reinforcement learning for enhancing indoor air quality and reducing energy consumption," Applied Energy, Elsevier, vol. 390(C).
    26. Gong, Quan & Lu, Lin & Chen, Jianheng, 2024. "Progress in radiative cooling materials for urban skin: Achievements in scalability, durability, color modulation, and intelligent thermal regulation," Renewable Energy, Elsevier, vol. 237(PB).
    27. Li, Hao & Zhang, Ji & Liu, Xiaohua & Zhang, Tao, 2022. "Comparative investigation of energy-saving potential and technical economy of rooftop radiative cooling and photovoltaic systems," Applied Energy, Elsevier, vol. 328(C).
    28. Yu, F.W. & Chan, K.T., 2008. "Optimization of water-cooled chiller system with load-based speed control," Applied Energy, Elsevier, vol. 85(10), pages 931-950, October.
    29. Lei, Lei & Wu, Bing & Fang, Xin & Chen, Li & Wu, Hao & Liu, Wei, 2023. "A dynamic anomaly detection method of building energy consumption based on data mining technology," Energy, Elsevier, vol. 263(PA).
    30. Zhang, Kai & Zhao, Dongliang & Yin, Xiaobo & Yang, Ronggui & Tan, Gang, 2018. "Energy saving and economic analysis of a new hybrid radiative cooling system for single-family houses in the USA," Applied Energy, Elsevier, vol. 224(C), pages 371-381.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Liang, Yan & Zhang, Weiyi & Yang, Haibing & Liu, Junwei & Zhou, Yifan & Cui, Hongzhi & Yan, Jinyue, 2025. "Bridging the critical gaps of radiative sky cooling: From lab to applications," Renewable and Sustainable Energy Reviews, Elsevier, vol. 222(C).
    2. Gong, Quan & Lu, Lin & Chen, Jianheng, 2024. "Progress in radiative cooling materials for urban skin: Achievements in scalability, durability, color modulation, and intelligent thermal regulation," Renewable Energy, Elsevier, vol. 237(PB).
    3. Xueke Wu & Jinlei Li & Fei Xie & Xun-En Wu & Siming Zhao & Qinyuan Jiang & Shiliang Zhang & Baoshun Wang & Yunrui Li & Di Gao & Run Li & Fei Wang & Ya Huang & Yanlong Zhao & Yingying Zhang & Wei Li & , 2024. "A dual-selective thermal emitter with enhanced subambient radiative cooling performance," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    4. Bian, Yuexin & Schmidt, Oliver & Shi, Yuanyuan, 2026. "Operator learning for energy-efficient building ventilation control with computational fluid dynamics simulation of a real-world classroom," Applied Energy, Elsevier, vol. 404(C).
    5. Yan, Tian & Xu, Dawei & Meng, Jing & Xu, Xinhua & Yu, Zhongyi & Wu, Huijun, 2024. "A review of radiative sky cooling technology and its application in building systems," Renewable Energy, Elsevier, vol. 220(C).
    6. Liu, Shuo & Liu, Xiaohua & Zhang, Tao & Wang, Chaoliang & Liu, Wei, 2024. "Joint optimization for temperature and humidity independent control system based on multi-agent reinforcement learning with cooperative mechanisms," Applied Energy, Elsevier, vol. 375(C).
    7. Chi, Fang'ai & Wu, Yun, 2025. "Residential buildings integrated with SPVP and RCL towards energy saving," Renewable and Sustainable Energy Reviews, Elsevier, vol. 212(C).
    8. Wang, Ze-Ye & Wu, Xian & Qu, Ming-Liang & Fan, Li-Wu & Yu, Zi-Tao & Chen, Shu-Qin & Ge, Jian & Wang, Liang & Dai, Sheng-Juan, 2025. "A field test and evaluation of radiative cooling performance as applied on the sidewall surfaces of residential buildings in China," Applied Energy, Elsevier, vol. 379(C).
    9. Forte, Davide & Belotti, Claudio & Pattelli, Lorenzo & Morciano, Matteo & Chiavazzo, Eliodoro & Asinari, Pietro & Fasano, Matteo, 2025. "Modeling of daytime radiative cooling enhanced vapor-compression refrigeration systems," Energy, Elsevier, vol. 340(C).
    10. Gong, Quan & Lu, Lin & Chen, Jianheng, 2023. "Design and performance investigation of a novel self-adaptive radiative cooling module for thermal regulation in buildings," Applied Energy, Elsevier, vol. 352(C).
    11. Li, Ze & Chen, Jianheng & Wang, Chuyao & Wang, Wenqi & Fu, Yang & Chen, Xu & Zhang, Rui & Pan, Aiqiang & Ho, Tsz Chung & Lin, Kaixin & Liang, Lin & Tso, Chi Yan, 2025. "Enhancing sustainable urban environments in China: Daytime radiative cooling for building energy efficiency and heat island mitigation," Applied Energy, Elsevier, vol. 393(C).
    12. Wang, Shuoyan & Yang, Liu & Liu, Yan & Pang, Jia & Yang, Liping & Dou, Mei, 2025. "Full-scale experimental study of the surface cooling effect of prefabricated buildings utilizing passive radiative cooling under real operating conditions," Energy, Elsevier, vol. 328(C).
    13. Uyanga, Kindness A. & Fan, Wenxiao & Han, Jie, 2025. "Advancing passive radiative cooling technology for green buildings: The potential and challenges of hydrogels," Renewable and Sustainable Energy Reviews, Elsevier, vol. 222(C).
    14. Pirvaram, Atousa & Talebzadeh, Nima & Leung, Siu Ning & O'Brien, Paul G., 2022. "Radiative cooling for buildings: A review of techno-enviro-economics and life-cycle assessment methods," Renewable and Sustainable Energy Reviews, Elsevier, vol. 162(C).
    15. Razzano, Giuseppe & Brandi, Silvio & Piscitelli, Marco Savino & Capozzoli, Alfonso, 2025. "Rule extraction from deep reinforcement learning controller and comparative analysis with ASHRAE control sequences for the optimal management of Heating, Ventilation, and Air Conditioning (HVAC) systems in multizone buildings," Applied Energy, Elsevier, vol. 381(C).
    16. Zhong, Fangliang & Calautit, John Kaiser & Wu, Yupeng, 2022. "Assessment of HVAC system operational fault impacts and multiple faults interactions under climate change," Energy, Elsevier, vol. 258(C).
    17. Bu, Fan & Yan, Da & Tan, Gang & Sun, Hongsan & An, Jingjing, 2023. "Acceleration algorithms for long-wavelength radiation integral in the annual simulation of radiative cooling in buildings," Renewable Energy, Elsevier, vol. 202(C), pages 255-269.
    18. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    19. Hu, Mingke & Zhao, Bin & Ao, Xianze & Feng, Junsheng & Cao, Jingyu & Su, Yuehong & Pei, Gang, 2019. "Experimental study on a hybrid photo-thermal and radiative cooling collector using black acrylic paint as the panel coating," Renewable Energy, Elsevier, vol. 139(C), pages 1217-1226.
    20. Xu, Nuo & Wang, Jiacheng & Cui, Yubo & Ren, Shenghao & Deng, Jiangbin & Gou, Qianzhi & Chen, Zhaoyu & Wang, Kaixin & Geng, Yang & Cui, Jiaxi & Li, Meng, 2024. "Butterfly wing-inspired microstructured film with high reflectivity for efficient passive radiative cooling," Renewable Energy, Elsevier, vol. 229(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:appene:v:402:y:2025:i:pa:s0306261925015934. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.