MAPPO-LCR: Multi-Agent Proximal Policy Optimization with Local Cooperation Reward in spatial public goods games

Imitation learning and conventional reinforcement learning struggle to capture strategic coupling in large spatial public goods games. This work introduces Multi-Agent Proximal Policy Optimization (MAPPO) into spatial public goods games to study learning-driven cooperation. Unlike independent PPO learners, MAPPO evaluates collective outcomes through a centralized critic with decentralized policy execution. This structure stabilizes learning dynamics and reveals clearer cooperation transition thresholds. Building on this framework, we examine MAPPO with a Local Cooperation Reward (MAPPO-LCR) to incorporate neighborhood-level cooperation signals. Local cooperative feedback strengthens spatial interactions and sharpens cooperation transitions near critical regimes. Extensive simulations and statistical analyses show that MAPPO is more stable than PPO, while MAPPO-LCR further improves robustness. These results clarify how deep multi-agent learning internalizes spatial cooperation mechanisms in structured social dilemmas.