A layered strategy updating mechanism for spatial public goods game with punishment

Cooperation is essential in human societies, yet fostering it among agents remains a longstanding challenge in evolutionary game theory. Among various mechanisms developed to sustain cooperation, punishment has emerged as an effective means to suppress defection and encourage cooperative behavior. Classical strategy updating rules such as the Fermi rule and Q-learning have therefore been widely applied in spatial public goods game to simulate the evolution of such mechanisms. However, these traditional mechanisms are often limited in their adaptability and struggle to maintain cooperation under harsh or dynamically changing conditions. To overcome these limitations, we propose an effective layered strategy updating mechanism for the spatial public goods game with punishment, which combines the network reciprocity of the Fermi rule with the self-learning capability of Q-learning by employing a layered process in which an agent first attempts to imitate a neighbor according to the Fermi rule, and, with the complementary probability, updates its strategy using the ϵ-greedy strategy from Q-learning. Simulation results demonstrate that the proposed layered strategy updating mechanism significantly improves cooperation compared to using either the Fermi rule or Q-learning alone. Lower punishment costs and higher fines are both found to promote higher levels of cooperation within the population. It is interesting to note that at equilibrium, the percentage of cooperators is constantly marginally greater than the percentage of punishers. In particular, increasing the punishment probability suppresses cooperation when the cost α exceeds the fine β and r is small, but promotes cooperation for larger r or when α is not greater than β. In addition, the influence of the discount factor γ on cooperation follows a non-monotonic pattern. Overall, this study provides a useful perspective on combining network reciprocity with self-learning and offers a potential foundation for further exploring complex evolutionary mechanisms.