Dual reinforcement learning synergy in resource allocation: Emergence of momentum strategy

Self-organized resource allocation is widespread and highly significant in both ecosystems and human societies. Given the diversity in decision-making policies, it is crucial to understand how these policies interact with each other. This study examines the influence of dual reinforcement learning policies (DRLP) on resource allocation in the minority game (MG). In a mixed population comprising Q-learning and classical policy subpopulations, a synergy effect emerges, resulting in superior resource allocation compared to either pure subpopulation. At the optimal mixing ratio, where resource allocation is maximized, a first-order phase transition occurs. Analysis reveals that internal and external synergy clusters in the subpopulation with Q-learning are crucial: the former enhance intra-synergy through synchronization and anti-synchronization within the subpopulation, while the latter prevent long-term resource underutilization due to the emergence of the classical financial market momentum strategy. However, this strategy causes trend reversals and lowers rewards for agents using it. We show that the frozen effect in either subpopulation is a necessary condition for synergy, in line with prior studies. Mathematical analyses were conducted to study intra-synergy through synchronization and anti-synchronization in the subpopulation with Q-learning. Overall, this study reveals the synergy mechanisms in DRLP-MG’s complex resource allocation dynamics, offering valuable practical insights.