Online hierarchical MPC–RL control for building heating without historical operating data

Building heating control without historical operating data remains challenging under dynamic electricity pricing. In this setting, model predictive control (MPC) is sensitive to model mismatch and calibration uncertainty, whereas reinforcement learning (RL) often suffers from unsafe early-stage exploration, unstable cold-start behavior, and limited interpretability. To address this deployment challenge, this paper proposes Safe Dual-Teacher Hierarchical MPC–RL (SDT-HRL), an online hierarchical control framework for practical building heating operation without historical operating data. The framework integrates rule-guided high-level target generation, risk-gated soft takeover between an MPC teacher and a learned low-level policy, and online physical adaptation through moving-horizon estimation and residual disturbance correction. The method is evaluated on the Building Optimization Testing Framework (BOPTEST), a standardized simulation-based benchmarking platform for building control, under benchmark-defined Peak (winter) and Typical (spring) scenarios. Unlike offline-trained or pre-identified baselines, SDT-HRL starts without offline pretraining, historical trajectory datasets, or hidden simulator states beyond the benchmark interface. In the Peak scenario, SDT-HRL maintains zero thermal discomfort while reducing operating cost and energy use by 3.6% and 3.3%, respectively, relative to MPC. In the Typical scenario, it reduces thermal discomfort and operating cost by 51.6% and 9.8%, respectively. Across the tested scenarios, SDT-HRL also outperforms representative state-of-the-art learning-based and hybrid baselines, including Safe-DRL and LearnAMR, and achieves a stronger overall cost–comfort trade-off. These results indicate that teacher-supported online adaptation can improve deployment reliability and overall heating-control performance under limited prior information.