Fast-apply deep autoregressive recurrent proximal policy optimization for controlling hot water systems

With the development of artificial intelligence technology, various intelligent algorithms are applied in building energy system optimization. Deep reinforcement learning (DRL) algorithms have garnered substantial attention from researchers. However, all current DRL-based control methods suffer from two problems. The first problem is that current DRL-based methods require an offline training process and therefore cannot be directly applied to houses. The offline training not only increases the waiting process for occupants but also creates the risk of degrading the occupant experience. The second problem is that current DRL-based methods do not continuously learn online. As a result of the second problem, the control methods are unable to consistently execute the optimal policy in the face of changing hot water demand habits. In this study, a fast-apply deep autoregressive recurrent proximal policy optimization (FDPPO) for controlling hot water systems is proposed. In practical systems, the FDPPO can be applied directly to houses without the need for occupants to wait. The proposed FDPPO can adapt to hot water demands that change over time through continuous online learning. In addition, the proposed FDPPO that applies the model-free reinforcement learning approach does not require modeling complex water heater models. The proposed FDPPO is evaluated by actual hot water demand data of over 55 weeks collected from two typical houses. The results show that the proposed FDPPO can save between 41.24% and 59.01% of energy consumption, all the while ensuring occupant comfort and safeguarding water hygiene.