IDEAS home Printed from https://ideas.repec.org/a/eee/apmaco/v510y2026ics0096300325004114.html

Reinforcement learning in evolutionary game theory: A brief review of recent developments

Author

Listed:
  • Xie, Kai
  • Szolnoki, Attila

Abstract

With the rapid progress of artificial intelligence, the integration of evolutionary game theory and reinforcement learning has become a hot research frontier in the last years. Evolutionary game theory provides a mathematical framework for depicting the strategy interaction among individuals, traditionally based on pre-defined, rule-based strategy update protocols. In contrast, reinforcement learning enables agents to adaptively select optimal actions through trial-and-error learning, hence better reflecting real-world decision-making. These complementary features create the foundation for their convergence. Our paper presents a didactic review of contemporary reinforcement learning applications in evolutionary game theory, focusing on those recently published works which open novel research paths to enrich our understanding of mutualistic cooperation. We summarize major concepts and terms, including the basic problem of collective cooperation, modeling of complex population dynamics, influence of algorithmic parameters, and the combination of deep learning. Finally, we discuss prospects for this interdisciplinary field, emphasizing the importance of intelligent learning through the lens of evolutionary game.

Suggested Citation

  • Xie, Kai & Szolnoki, Attila, 2026. "Reinforcement learning in evolutionary game theory: A brief review of recent developments," Applied Mathematics and Computation, Elsevier, vol. 510(C).
  • Handle: RePEc:eee:apmaco:v:510:y:2026:i:c:s0096300325004114
    DOI: 10.1016/j.amc.2025.129685
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0096300325004114
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.amc.2025.129685?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Xie, Kai & Szolnoki, Attila, 2025. "Reputation in public goods cooperation under double Q-learning protocol," Chaos, Solitons & Fractals, Elsevier, vol. 196(C).
    2. Raphael Koster & Miruna Pîslar & Andrea Tacchetti & Jan Balaguer & Leqi Liu & Romuald Elie & Oliver P. Hauser & Karl Tuyls & Matt Botvinick & Christopher Summerfield, 2025. "Deep reinforcement learning can promote sustainable human behaviour in a common-pool resource problem," Nature Communications, Nature, vol. 16(1), pages 1-13, December.
    3. Zheng, Guozhong & Zhang, Jiqiang & Deng, Shengfeng & Cai, Weiran & Chen, Li, 2024. "Evolution of cooperation in the public goods game with Q-learning," Chaos, Solitons & Fractals, Elsevier, vol. 188(C).
    4. Xie, Kai & Liu, Tingjin, 2024. "The regulation of good and evi promotes cooperation in public goods game," Applied Mathematics and Computation, Elsevier, vol. 478(C).
    5. Xie, Kai & Liu, Xingwen & Chen, Hao & Yang, Jun, 2022. "Preferential selection and expected payoff drive cooperation in spatial voluntary public goods game," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 605(C).
    6. Qi Shi & Shilin Xiao & Qionglin Dai & Haihong Li & Junzhong Yang, 2022. "Spatial structure might impede cooperation in evolutionary games with reinforcement learning," International Journal of Modern Physics C (IJMPC), World Scientific Publishing Co. Pte. Ltd., vol. 33(12), pages 1-13, December.
    7. Bai, Pengzhou & Qiang, Bingzhuang & Zou, Kuan & Huang, Changwei, 2024. "Preferential selection based on adaptive attractiveness induce by reinforcement learning promotes cooperation," Chaos, Solitons & Fractals, Elsevier, vol. 180(C).
    8. Ling, Ting & Li, Zhang & Feng, Minyu & Szolnoki, Attila, 2025. "Supervised cooperation on interdependent public goods games," Applied Mathematics and Computation, Elsevier, vol. 492(C).
    9. Zou, Kuan & Huang, Changwei, 2024. "Incorporating reputation into reinforcement learning can promote cooperation on hypergraphs," Chaos, Solitons & Fractals, Elsevier, vol. 186(C).
    10. Ding, Hong & Zhang, Geng-shun & Wang, Shi-hao & Li, Juan & Wang, Zhen, 2019. "Q-learning boosts the evolution of cooperation in structured population by involving extortion," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 536(C).
    11. Lv, Shaojie & Li, Jiaying & Zhao, Changheng, 2023. "The evolution of cooperation in voluntary public goods game with shared-punishment," Chaos, Solitons & Fractals, Elsevier, vol. 172(C).
    12. Jorgen W. Weibull, 1997. "Evolutionary Game Theory," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262731215, December.
    13. Kevin R. McKee & Andrea Tacchetti & Michiel A. Bakker & Jan Balaguer & Lucy Campbell-Gillingham & Richard Everett & Matthew Botvinick, 2023. "Scaffolding cooperation in human groups with deep reinforcement learning," Nature Human Behaviour, Nature, vol. 7(10), pages 1787-1796, October.
    14. Lee, Hsuan-Wei & Weng, Yi-Ning, 2025. "Granular Q-learning adaptation boosts collective welfare in multi-agent Prisoner’s Dilemma," Chaos, Solitons & Fractals, Elsevier, vol. 199(P1).
    15. Zhang, Lan & Li, Yuqin & Xie, Yuan & Feng, Yuee & Huang, Changwei, 2025. "The combined effects of conformity and reinforcement learning on the evolution of cooperation in public goods games," Chaos, Solitons & Fractals, Elsevier, vol. 193(C).
    16. Xie, Kai & Liu, Yaojun & Liu, Tingjin, 2024. "Unveiling the masks: Deception and reputation in spatial prisoner’s dilemma game," Chaos, Solitons & Fractals, Elsevier, vol. 186(C).
    17. Li, Yipeng & Hu, Xiangyue & Jin, Xing & Zhang, Huizhen & Yang, Jiajia & Wang, Zhen, 2025. "Environmental information perception enhances cooperation in stochastic public goods games via Q-learning," Applied Mathematics and Computation, Elsevier, vol. 504(C).
    18. Xuzhen Zhu & Xin Su & Jinming Ma & Hui Tian & RunRan Liu, 2019. "Evolutionary Cooperation in Networked Public Goods Game with Dependency Groups," Complexity, Hindawi, vol. 2019, pages 1-8, October.
    19. repec:fth:iniesr:487 is not listed on IDEAS
    20. Yang, Zhengzhi & Zheng, Lei & Perc, Matjaž & Li, Yumeng, 2024. "Interaction state Q-learning promotes cooperation in the spatial prisoner's dilemma game," Applied Mathematics and Computation, Elsevier, vol. 463(C).
    21. Wang, Pai & Yang, Zhihu, 2024. "The double-edged sword effect of conformity on cooperation in spatial Prisoner’s Dilemma Games with reinforcement learning," Chaos, Solitons & Fractals, Elsevier, vol. 187(C).
    22. Wang, Chengjie & Deng, Juan & Zhao, Hui & Li, Li, 2024. "Effect of Q-learning on the evolution of cooperation behavior in collective motion: An improved Vicsek model," Applied Mathematics and Computation, Elsevier, vol. 482(C).
    23. Wang, Xianjia & Yang, Zhipeng & Liu, Yanli & Chen, Guici, 2023. "A reinforcement learning-based strategy updating model for the cooperative evolution," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 618(C).
    24. Zhang, Huizhen & An, Tianbo & Yan, Pingping & Hu, Kaipeng & An, Jinjin & Shi, Lijuan & Zhao, Jian & Wang, Jingrui, 2024. "Exploring cooperative evolution with tunable payoff’s loners using reinforcement learning," Chaos, Solitons & Fractals, Elsevier, vol. 178(C).
    25. Zhu, Yuying & Xing, Bohua & Xia, Chengyi, 2025. "Q-learning update with second-order reputation promotes the evolution of trust within structured populations," Chaos, Solitons & Fractals, Elsevier, vol. 199(P1).
    26. Wang, Qiushuang & Chen, Xiaojie & Szolnoki, Attila, 2025. "Coevolutionary dynamics of feedback-evolving games in structured populations," Chaos, Solitons & Fractals, Elsevier, vol. 193(C).
    27. Qian, Yinuo & Zhao, Dawei & Xia, Chengyi, 2025. "Disease transmission in dynamic social networks constructed by reinforcement learning-driven preventive game," Chaos, Solitons & Fractals, Elsevier, vol. 199(P1).
    28. Yang, Yujin & Zhao, Dawei & Wang, Juan, 2025. "Evolution of cooperation in spatial public goods games driven by reinforcement learning and environmental feedback," Chaos, Solitons & Fractals, Elsevier, vol. 199(P1).
    29. Lv, Shaojie & Li, Jiaying & Zhao, Changheng, 2025. "Reinforcement learning in spatial public goods games with environmental feedbacks," Chaos, Solitons & Fractals, Elsevier, vol. 195(C).
    30. Ueda, Masahiko, 2023. "Memory-two strategies forming symmetric mutual reinforcement learning equilibrium in repeated prisoners’ dilemma game," Applied Mathematics and Computation, Elsevier, vol. 444(C).
    31. Zhang, Qianwei & Zhang, Xinran, 2025. "Q-learning driven cooperative evolution with dual-reputation incentive mechanisms," Applied Mathematics and Computation, Elsevier, vol. 507(C).
    32. Si, Zehua & Ito, Takayuki, 2025. "Beyond cyclic dominance: Reinforcement learning promotes cooperation in the spatial rock–paper–scissors game," Chaos, Solitons & Fractals, Elsevier, vol. 199(P1).
    33. Shen, Yong & Ma, Yujie & Kang, Hongwei & Sun, Xingping & Chen, Qingyi, 2024. "Learning and propagation: Evolutionary dynamics in spatial public goods games through combined Q-learning and Fermi rule," Chaos, Solitons & Fractals, Elsevier, vol. 187(C).
    34. Usui, Yuki & Ueda, Masahiko, 2021. "Symmetric equilibrium of multi-agent reinforcement learning in repeated prisoner’s dilemma," Applied Mathematics and Computation, Elsevier, vol. 409(C).
    35. Ding, Zhen-Wei & Zhang, Ji-Qiang & Zheng, Guo-Zhong & Cai, Wei-Ran & Cai, Chao-Ran & Chen, Li & Wang, Xu-Ming, 2024. "Emergence of anti-coordinated patterns in snowdrift game by reinforcement learning," Chaos, Solitons & Fractals, Elsevier, vol. 184(C).
    36. Jia, Danyang & Li, Tong & Zhao, Yang & Zhang, Xiaoqin & Wang, Zhen, 2022. "Empty nodes affect conditional cooperation under reinforcement learning," Applied Mathematics and Computation, Elsevier, vol. 413(C).
    37. Lin, Jiaying & Long, Pinduo & Liang, Jinfeng & Dai, Qionglin & Li, Haihong & Yang, Junzhong, 2025. "The coevolution of cooperation: Integrating Q-learning and occasional social interactions in evolutionary games," Chaos, Solitons & Fractals, Elsevier, vol. 194(C).
    38. Shen, Shaofei & Zhang, Xuejun & Xu, Aobo & Duan, Taisen, 2024. "An adaptive exploration mechanism for Q-learning in spatial public goods games," Chaos, Solitons & Fractals, Elsevier, vol. 189(P1).
    39. Young, H. Peyton, 2009. "Learning by trial and error," Games and Economic Behavior, Elsevier, vol. 65(2), pages 626-643, March.
    40. Ding, Zhen-Wei & Zheng, Guo-Zhong & Cai, Chao-Ran & Cai, Wei-Ran & Chen, Li & Zhang, Ji-Qiang & Wang, Xu-Ming, 2023. "Emergence of cooperation in two-agent repeated games with reinforcement learning," Chaos, Solitons & Fractals, Elsevier, vol. 175(P1).
    41. Takahiro Ezaki & Naoki Masuda, 2017. "Reinforcement learning account of network reciprocity," PLOS ONE, Public Library of Science, vol. 12(12), pages 1-8, December.
    42. Geng, Yini & Liu, Yifan & Lu, Yikang & Shen, Chen & Shi, Lei, 2022. "Reinforcement learning explains various conditional cooperation," Applied Mathematics and Computation, Elsevier, vol. 427(C).
    43. Zhang, Gui & Xiong, Xiaojin & Pi, Bin & Feng, Minyu & Perc, Matjaž, 2025. "Spatial public goods games with queueing and reputation," Applied Mathematics and Computation, Elsevier, vol. 505(C).
    44. Xie, Kai & Liu, Xingwen & Wang, Huazhang & Jiang, Yulian, 2023. "Multi-heterogeneity public goods evolutionary game on lattice," Chaos, Solitons & Fractals, Elsevier, vol. 172(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Wu, Yu’e & Liu, Zeyun & Wang, Xinyu & Yin, Anran & Zhang, Shuhua, 2026. "Research on the diffusion of low-carbon technologies through trilateral evolutionary game based on incentive mechanisms," Applied Mathematics and Computation, Elsevier, vol. 516(C).
    2. Han, Minglong & Liu, Yupeng, 2026. "Dynamic pricing mechanisms for generative artificial intelligence models across heterogeneous scenarios: An evolutionary game and complex network approach," Applied Mathematics and Computation, Elsevier, vol. 516(C).
    3. Li, Chanchan & Yang, Zhaoqilin & Jia, Wensheng & Zhao, Hongxin & Wang, Xin, 2026. "Multi-expert policy distillation guided proximal policy optimization for efficient cooperation emergence in spatial public goods games," Chaos, Solitons & Fractals, Elsevier, vol. 202(P1).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kang, Hongwei & Jiang, Chao & Shen, Yong & Sun, Xingping & Chen, Qingyi, 2025. "Neighbor-aware reinforcement learning fosters cooperation in spatial public goods games," Chaos, Solitons & Fractals, Elsevier, vol. 199(P3).
    2. Xie, Kai & Szolnoki, Attila, 2025. "Reputation in public goods cooperation under double Q-learning protocol," Chaos, Solitons & Fractals, Elsevier, vol. 196(C).
    3. Zhang, Yongqiang & Zheng, Zehao & Zhang, Xiaoming & Ma, Jinlong, 2025. "A layered strategy updating mechanism for spatial public goods game with punishment," Chaos, Solitons & Fractals, Elsevier, vol. 201(P2).
    4. An, Tianbo & Zhang, Huizhen & Zhang, Zhanshuo & Liu, Guanghui & Li, Jiayu & Chen, Liangyu & Wang, Zhen, 2025. "Cooperation dynamics driven by reinforcement learning with interactive diversity in structured populations," Chaos, Solitons & Fractals, Elsevier, vol. 201(P2).
    5. Yang, Yujin & Zhao, Dawei & Wang, Juan, 2025. "Evolution of cooperation in spatial public goods games driven by reinforcement learning and environmental feedback," Chaos, Solitons & Fractals, Elsevier, vol. 199(P1).
    6. Lin, Jiaying & Yang, Junzhong, 2025. "Emotion-coupled Q-learning with cognitive bias enhances cooperation in evolutionary prisoner’s dilemma games," Chaos, Solitons & Fractals, Elsevier, vol. 200(P1).
    7. Zhang, Qianwei & Zhang, Xinran, 2025. "Q-learning driven cooperative evolution with dual-reputation incentive mechanisms," Applied Mathematics and Computation, Elsevier, vol. 507(C).
    8. Zheng, Guozhong & Zhang, Jiqiang & Deng, Shengfeng & Cai, Weiran & Chen, Li, 2024. "Evolution of cooperation in the public goods game with Q-learning," Chaos, Solitons & Fractals, Elsevier, vol. 188(C).
    9. Huang, Yijie, 2025. "The evolution of cooperation in multi-games with reinforcement learning," Chaos, Solitons & Fractals, Elsevier, vol. 201(P2).
    10. Wang, Chengjie & Deng, Juan & Zhao, Hui & Li, Li, 2024. "Effect of Q-learning on the evolution of cooperation behavior in collective motion: An improved Vicsek model," Applied Mathematics and Computation, Elsevier, vol. 482(C).
    11. Xie, Kai & Liu, Yaojun & Liu, Tingjin, 2024. "Unveiling the masks: Deception and reputation in spatial prisoner’s dilemma game," Chaos, Solitons & Fractals, Elsevier, vol. 186(C).
    12. Wang, Weining & Shang, Lihui & Wu, Yipeng & Hu, Mingjian & Wang, Weiyu, 2025. "Bio-inspired mechanism promotes cooperation in spatial public goods games," Chaos, Solitons & Fractals, Elsevier, vol. 200(P3).
    13. Li, Yipeng & Hu, Xiangyue & Jin, Xing & Zhang, Huizhen & Yang, Jiajia & Wang, Zhen, 2025. "Environmental information perception enhances cooperation in stochastic public goods games via Q-learning," Applied Mathematics and Computation, Elsevier, vol. 504(C).
    14. Yue, Hongyu & Xiong, Xiaojin & Feng, Minyu & Szolnoki, Attila, 2025. "Dynamic evolution of cooperation based on adaptive reputation threshold and game transition," Chaos, Solitons & Fractals, Elsevier, vol. 199(P1).
    15. Liu, Xinyu & Jin, Wei & Chen, Guanrong & Tang, Changbing & Qian, Youhua & Jin, Weifeng, 2025. "Small groups nurturing collective wisdom: The punishment-prediction reinforcement learning mechanism for multi-group cooperation," Chaos, Solitons & Fractals, Elsevier, vol. 201(P1).
    16. Wang, Xiaonan & Lu, Gang & Guo, Peng, 2025. "Modeling spatial public goods games with differentiated strategies and asymmetric payoff allocation," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 678(C).
    17. Shen, Shaofei & Zhang, Xuejun & Xu, Aobo & Duan, Taisen, 2024. "An adaptive exploration mechanism for Q-learning in spatial public goods games," Chaos, Solitons & Fractals, Elsevier, vol. 189(P1).
    18. Li, Chanchan & Yang, Zhaoqilin & Jia, Wensheng & Zhao, Hongxin & Wang, Xin, 2026. "Multi-expert policy distillation guided proximal policy optimization for efficient cooperation emergence in spatial public goods games," Chaos, Solitons & Fractals, Elsevier, vol. 202(P1).
    19. Lee, Hsuan-Wei & Weng, Yi-Ning, 2025. "Granular Q-learning adaptation boosts collective welfare in multi-agent Prisoner’s Dilemma," Chaos, Solitons & Fractals, Elsevier, vol. 199(P1).
    20. Lv, Shaojie & Li, Jiaying & Zhao, Changheng, 2025. "Reinforcement learning in spatial public goods games with environmental feedbacks," Chaos, Solitons & Fractals, Elsevier, vol. 195(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:apmaco:v:510:y:2026:i:c:s0096300325004114. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/applied-mathematics-and-computation .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.