Heterogeneous Exploration and Double-Critic Transfer Reinforcement Learning for Sustainable Cross-Domain Energy Management in Smart Buildings

The integration of distributed energy resources (DERs) has enhanced the operational flexibility and complexity of smart building energy management, which is crucial to urban sustainable development. However, the limitations of strategy applicability across different environments and lengthy development cycles pose significant challenges for energy management. To address this, this paper proposes a transferred multi-thread deep reinforcement learning (TMDRL) framework for the cross-domain energy management of smart buildings. Firstly, a source-domain heterogeneous exploration architecture based on multi-thread deep reinforcement learning (DRL) is proposed. A transferable source-domain knowledge base is constructed to enhance the generalization ability of pre-trained strategies. Secondly, a decoupled double-critic optimization mechanism is designed to mitigate policy evaluation bias during cross-domain transfer. Finally, simulations using real-world datasets from different times and areas are conducted. The results show that compared to A3C, DDPG, and SAC, the proposed TMDRL framework reduces total costs by 32.77%, 18.14%, and 37.24%, while improving convergence efficiency by 29.55%, 22.89%, and 32.84%, respectively. The reduction in total cost and improvement in convergence efficiency demonstrate that the proposed TMDRL framework effectively saves energy and enhances the utilization of renewable energy, proving the sustainable benefits of smart building energy management across domains.