Boiler combustion optimization via offline reinforcement learning with an ensemble high-dimensional environment

Reinforcement learning (RL) offers a promising framework for multi-objective optimization in the complex and highly dynamic boiler combustion process. However, existing RL methods often suffer from accumulated model errors and over-reliance on simplified low-dimensional models, failing to capture real-world combustion dynamics. To address these challenges, this study develops a high-dimensional digital twin environment. In addition, an ensemble modeling strategy that integrates the tree-structured Parzen estimator with gradient normalization is proposed to mitigate gradient imbalance in multi-task learning and to construct a robust high-dimensional dynamic predictive model. Building upon this digital twin environment, a behavioral proximal policy optimization (BPPO) agent is trained entirely in an offline manner. The proposed framework effectively addresses the distribution shift issue in offline RL without additional constraints, and enables fully offline, safe and stable policy learning for multi-objective industrial boiler combustion optimization without any direct interaction with the physical boiler system. The performance of this framework is validated on real historical operational data from an industrial biomass circulating fluidized bed boiler. Simulation experiments show that, in single-step optimization, the BPPO agent increases thermal efficiency by 0.15% and reduces NOx emissions by 5.52 mg/m3, outperforming representative offline RL baselines in balancing these objectives. In multi-step optimization simulations, benefits are further amplified, with thermal efficiency increased by 0.40% and NOx emissions reduced by 13.50 mg/m3. These results demonstrate a safe, efficient, and generalizable solution for optimizing complex high-dimensional industrial boiler combustion processes, and support the practical industrial deployment of offline reinforcement learning in combustion control.