Learning and propagation: Evolutionary dynamics in spatial public goods games through combined Q-learning and Fermi rule

Propagation is crucial for acquiring information, and learning involves deep information processing. Imitation dynamics, commonly used in spatial public goods games, represents strategy propagation in society, while learning dynamics enables agents to self-learn through environmental interactions. In this paper, Q-learning and the Fermi update rule are used to compare differences between learning dynamics and imitation dynamics in simulation experiments. Our study finds that Q-learning is continuously updating the Q-table during its evolution, forming a heterogeneous gain matrix. However, Q-learning’s interaction with the environment is indirect, with almost no network reciprocity, and cannot form clusters in contrast to the Fermi update rule. The Fermi update rule focus only on immediate payoffs. Furthermore, we combine imitation dynamics and learning dynamics, integrating the advantages of both. Defectors and cooperators form a special semi-stable checkerboard-like state. The update rules, the number of states, and the configuration of conversion actions as strategies have led to this special structure. Moreover, we found that parameter choices in the new evolutionary dynamics affect the percentage of cooperation and evolutionary convergence. Overall, this paper offers new insights into learning dynamics and imitation dynamics and provides a foundation for integrating complex mechanisms of evolution.