Q-learning update with second-order reputation promotes the evolution of trust within structured populations

The Q-learning characterizes how individuals adjust strategies to adapt to the environment, while its interaction with the reputation evaluation in trust games remains underexplored despite their coexistence in real systems. This paper combines the Q-learning algorithm with the second-order social norms to analyze the evolution of altruistic behaviors within the framework of networked N-player trust game. Given different dilemma intensities, the Q-learning mechanism yields richer steady-state strategy distributions than the traditional Fermi rule, which only considers immediate payoffs and exhibits inhibition of trustworthy behaviors. Then, based on four second-order reputation evaluation rules, it is shown that, even when the dilemma intensity is large, the Shunning rule still can help to improve global wealth. In addition, we analyze the role of critical parameters in reinforcement learning, including the learning coefficient, reward discount factor and exploration probability, thus determining the effectiveness of the proposed mode. Sensitivity analysis reveals that large learning rates and discount factor contribute to increasing global wealth, while a smaller exploration rate is crucial for maintaining trust and wealth accumulation. The proposed strategy update model yields greater adaptive flexibility in complex environments and provides a new perspective for understanding the maintenance and evolution of trust behaviors in complex social systems.