Deep reinforcement learning-based residential building energy management incorporating power-to-heat technology for building electrification

Building electrification considerably contributes to reducing global energy consumption and carbon emissions, necessitating efficient energy management systems. Although heat pumps (HPs) are promising for power-to-heat (P2H) applications, utilizing single HP systems could limit economic performance and flexibility, particularly in multi-family buildings. Additionally, overestimation and underestimation biases in existing deep reinforcement learning (DRL) algorithms pose substantial challenges for developing optimal energy management strategies under various uncertainties. This study proposes a novel DRL-based residential building energy management system (RBEMS) integrating P2H technology for comprehensive management of electricity, heat, and carbon emissions. First, a two-stage HP system is employed, where a central-HP preheats water to an intermediate temperature, followed by distributed-HP that increases it to the target level, thereby improving system efficiency and enabling flexible control. Second, a novel dynamic action subset–twin delayed deep deterministic policy gradient (TD3) algorithm is proposed to efficiently bridge the gap between action selection and evaluation by selecting high-value subset actions near the optimum from one critic network and evaluating them with a secondary network. Case studies demonstrate that the proposed RBEMS increases total rewards by 20.1 % over a single HP system and 9 % over TD3, achieving lower energy costs, reduced carbon emissions, while ensuring user comforts.