Dual-scale optimization of integrated energy systems: a novel MAPPO-based approach for hybrid game equilibrium

With the accelerating global energy transition toward carbon neutrality, integrated energy systems face complex optimization challenges arising from renewable energy intermittency, diversified user demands, and the inherent time-scale misalignment between supplier pricing strategies and user responsive behaviors, creating multi-level game environments that existing methods struggle to address effectively, leading to insufficient equipment utilization and inadequate energy resource exploitation. To this end, this paper first establishes a comprehensive physical model incorporating electricity-gas-heat multi-energy coupling with diverse conversion equipment (electric boilers, heat pumps, gas boilers, CHP systems, and P2G facilities) and renewable energy sources. Second, Multi-Time-scale Graph-enhanced Multi-Agent Proximal Policy Optimization framework (MT-MAPPO) is proposed to address the hybrid game equilibrium problem in integrated energy system optimization. The framework constructs a comprehensive “vertical Stackelberg + horizontal competition” dual-game structure that combines supplier-user hierarchical relationships with peer-to-peer competitive dynamics among users. Third, Centralized Training with Decentralized Execution (CTDE) and policy clipping mechanisms are employed, while graph neural networks are integrated to model user transaction network topologies for enhanced state representation learning. Finally, a cross-time-scale collaborative optimization mechanism is developed through forward simulation and rolling optimization, enabling effective information transfer and value feedback between different temporal decision hierarchies. Experimental validation demonstrates that MT-MAPPO exhibits excellent performance in multi-agent energy system optimization, achieving good convergence speed and training stability. The average reward of MADQN and MADDPG is 5.4% and 16.2% lower than MAPPO.