Adaptive home energy management based on PI-DRA-PPO for integrated electricity, hydrogen, and heat storage systems

The Home Energy Management System (HEMS) is becoming increasingly vital for mitigating residential energy expenditures and maximizing the utilization of renewable energy sources through coordinated scheduling of various energy storage devices. Integrating hydrogen storage into HEMS has also garnered growing attention for its ability to enhance household energy management flexibility and reliability and support the long-term goal of net-zero energy homes (NZEHs). However, challenges such as uncertain electricity prices and intermittent photovoltaic (PV) outputs frequently result in inefficient management strategies, leading to PV curtailment, reduced energy efficiency, and unreliable energy supply. To this end, this paper proposes a novel data-informed Deep Reinforcement Learning (DRL) framework, termed Prediction Intervals Constrained Dynamic Reward Adaptive Proximity Policy Optimization (PI-DRA-PPO), to optimize household energy management by enhancing both energy supply reliability and economic efficiency. A probabilistic prediction module combining Bi-directional Gated Recursive Unit with Coded Engineering (BiGRU-CE) and the Kernel Density Estimation (KDE) is employed to construct reliable prediction intervals (PIs), which serve as safety constraints embedded within the state space of the PI-DRA-PPO algorithm. Additionally, the Hydrogen Energy Storage System (HESS) is strategically integrated with the Thermal Energy Storage System (TESS) and Battery Energy Storage System (BESS) within the proposed HEMS for the coordinated management of multi-timescale energy dynamics induced by the intermittency of renewable energy sources. Compared with basic PPO and other DRL methods, the proposed method can achieve up to a 62.86 % reduction in total energy costs over a representative four-month period covering all seasons. The approach improves the effectiveness of energy scheduling and strengthens the temporal and long-term reliability of HEMS in the face of dynamic energy environments.