LearnAMR: Learning-based adaptive model predictive control enhanced by reinforcement learning for optimizing energy flexibility in building energy systems incorporating demand-side management

Intelligent control offers significant potential to enhance the flexibility and overall performance of building energy systems (BES). However, conventional methods rely heavily on accurate models, limiting their adaptability to the dynamic and uncertain operational conditions of BES. Therefore, this work proposed a novel learning-based adaptive model predictive control (MPC) framework enhanced by reinforcement learning (RL), termed LearnAMR, to optimize energy flexibility of BES, incorporating demand-side management. Firstly, a flexible energy management strategy was formulated, integrating multiple flexibility resources such as energy storage, controllable electrical loads, and thermal inertia of buildings. The generalized flexibility indicators were established to evaluate energy efficiency, occupant satisfaction, and grid interaction of BES. Secondly, an RL-enhanced adaptive MPC algorithm was developed to solve the BES control problem, where RL was utilized to dynamically adjust the parameter combination of MPC to deal with model mismatches due to operational uncertainties. Thirdly, a sparse neural network-based thermal dynamics modeling approach was designed to accurately capture the characteristics of building thermal response. The trained model was subsequently transformed into a mixed-integer linear programming representation, enabling its seamless integration into the optimization framework and facilitating real-time, lightweight prediction of thermal responses. A case study building was presented to demonstrate the performance of the proposed methodology. The results showed that the LearnAMR method could reduce energy consumption by 12.1 % while achieving the highest cost savings compared to the other baseline algorithms. Moreover, the proposed approach outperformed conventional methods in photovoltaic utilization, grid independence, and computational efficiency.