Hierarchical multi-scale co-interactive deep reinforcement learning for evolutionary games in dynamic complex networks

Understanding how cooperation emerges and persists in dynamic complex networks remains a central challenge in evolutionary game theory (EGT). In many real-world systems, agents adapt their strategies while the underlying interaction structure simultaneously evolves, leading to strong non-stationarity that undermines classical game-theoretic and reinforcement learning (RL) approaches. Existing methods typically decouple strategic learning from network adaptation or rely on static control mechanisms, resulting in fragile cooperation under environmental or incentive perturbations. In this work, we propose a Hierarchical Multi-Scale Co-Interactive Deep Reinforcement Learning (HiDRL) framework to address the joint evolution of agent strategies and network structure. The framework integrates hybrid learning system with network evolution mechanism through a hierarchical control architecture that operates across distinct temporal scales. At the lower level, agents learn strategic behaviors through deep reinforcement learning, while at higher levels, policy reasoning and hyperparameter evolution mechanisms regulate interaction topology and learning dynamics, enabling the coordinated adaptation of both behavior and structure. We evaluate the proposed framework in evolutionary prisoner’s dilemma (PD) games on dynamic complex networks. Experiments demonstrate that HiDRL sustains high levels of cooperation across a wide range of dilemma intensities and further show that hierarchical control stabilizes cooperation by mitigating phase transitions and buffering the system from incentive shocks. These findings suggest that hierarchical, multi-scale learning provides a general and robust paradigm for studying cooperation in co-evolving socio-technical systems, where both agent behavior and interaction structure are inherently adaptive.