Q-learning-driven migration based on neighborhood composition promotes cooperation in spatial prisoner's dilemma

In spatial evolutionary prisoner's dilemma games, migration serves as a key mechanism regulating cooperative dynamics. Existing migration models generally rely on simplified environmental perception and pre-set decision rules, leaving two critical questions underexplored: how individuals perceive complex neighborhood information, and how they adaptively adjust their migration strategies through experiential learning. In this work, we propose a Q-learning-driven migration mechanism that aggregates raw neighborhood configurations into compact “Meta States” with explicit game-theoretic implications, and learns optimal migration policies for each state via Q-learning. Simulation results demonstrate that the proposed mechanism effectively enhances cooperation, with the most pronounced promotion observed at intermediate population densities and moderate imitation probabilities. Analysis of the underlying dynamics reveals that Q-learning enables individuals to actively aggregate into dense, stable cooperative clusters, thereby structurally reinforcing cooperation. Furthermore, the mechanism's effect is robustly observed under deterministic strategy updating (best-take-over rule), but exhibits a density-dependent reversal under stochastic updating (Fermi rule), highlighting the intricate interplay between migration and social learning. Comparative experiments further validate the effectiveness of our state-partitioning approach in capturing strategically relevant environmental information.