IDEAS home Printed from https://ideas.repec.org/a/eee/chsofr/v195y2025ics0960077925003091.html

Reinforcement learning in spatial public goods games with environmental feedbacks

Author

Listed:
  • Lv, Shaojie
  • Li, Jiaying
  • Zhao, Changheng

Abstract

The feedback between strategy and environment is ubiquitous in nature and human society, which has been receiving increasing attention from researchers. Meanwhile, Q-learning allows players to explore the optimal strategy by interacting with the environment. In this paper, we introduce the Q-learning into the spatial public goods game with environmental feedbacks. The simulation results show that the environmental feedback can promote cooperation. The increase of synergy coefficient r and strength of the environmental feedback α is beneficial for the evolution of cooperation. The effects of discount factor γ on the cooperation level of the population are non-monotonic. When r or α is low, the high values of γ can promote the emergence of cooperation. However, with the increase of r and α, the low values of γ are more favorable to the evolution of cooperation.

Suggested Citation

  • Lv, Shaojie & Li, Jiaying & Zhao, Changheng, 2025. "Reinforcement learning in spatial public goods games with environmental feedbacks," Chaos, Solitons & Fractals, Elsevier, vol. 195(C).
    • Handle: RePEc:eee:chsofr:v:195:y:2025:i:c:s0960077925003091
    DOI: 10.1016/j.chaos.2025.116296
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0960077925003091
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.chaos.2025.116296?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Andreoni, James, 1988. "Why free ride? : Strategies and learning in public goods experiments," Journal of Public Economics, Elsevier, vol. 37(3), pages 291-304, December.
    2. Dolgopolov, Arthur, 2024. "Reinforcement learning in a prisoner's dilemma," Games and Economic Behavior, Elsevier, vol. 144(C), pages 84-103.
    3. Andrew R. Tilman & Joshua B. Plotkin & Erol Akçay, 2020. "Evolutionary games with environmental feedbacks," Nature Communications, Nature, vol. 11(1), pages 1-11, December.
    4. Valerio Capraro & Roberto Di Paolo & Matjaz Perc & Veronica Pizziol, 2024. "Language-based game theory in the age of artificial intelligence," Papers 2403.08944, arXiv.org.
    5. Quan, Ji & Li, Haoze & Zhang, Man & Wang, Xianjia, 2024. "Cooperation dynamics in nonlinear spatial public goods games with endogenous synergy and discounting feedback," Chaos, Solitons & Fractals, Elsevier, vol. 178(C).
    6. Ma, Xiaojian & Quan, Ji & Wang, Xianjia, 2023. "Evolution of cooperation with nonlinear environment feedback in repeated public goods game," Applied Mathematics and Computation, Elsevier, vol. 452(C).
    7. Yang, Zhengzhi & Zheng, Lei & Perc, Matjaž & Li, Yumeng, 2024. "Interaction state Q-learning promotes cooperation in the spatial prisoner's dilemma game," Applied Mathematics and Computation, Elsevier, vol. 463(C).
    8. Simon Gachter & Ernst Fehr, 2000. "Cooperation and Punishment in Public Goods Experiments," American Economic Review, American Economic Association, vol. 90(4), pages 980-994, September.
    9. Ding, Rui & Wang, Xianjia & Liu, Yang & Zhao, Jinhua & Gu, Cuiling, 2023. "Evolutionary games with environmental feedbacks under an external incentive mechanism," Chaos, Solitons & Fractals, Elsevier, vol. 169(C).
    10. Nikiforakis, Nikos, 2010. "Feedback, punishment and cooperation in public good experiments," Games and Economic Behavior, Elsevier, vol. 68(2), pages 689-702, March.
    11. Ding, Zhen-Wei & Zhang, Ji-Qiang & Zheng, Guo-Zhong & Cai, Wei-Ran & Cai, Chao-Ran & Chen, Li & Wang, Xu-Ming, 2024. "Emergence of anti-coordinated patterns in snowdrift game by reinforcement learning," Chaos, Solitons & Fractals, Elsevier, vol. 184(C).
    12. Jia, Danyang & Li, Tong & Zhao, Yang & Zhang, Xiaoqin & Wang, Zhen, 2022. "Empty nodes affect conditional cooperation under reinforcement learning," Applied Mathematics and Computation, Elsevier, vol. 413(C).
    13. Lv, Shaojie & Li, Jiaying & Zhao, Changheng, 2024. "Fixation of cooperation in evolutionary games with environmental feedbacks," Applied Mathematics and Computation, Elsevier, vol. 482(C).
    14. Cao, Lixuan & Wu, Bin, 2021. "Eco-evolutionary dynamics with payoff-dependent environmental feedback," Chaos, Solitons & Fractals, Elsevier, vol. 150(C).
    15. Geng, Yini & Liu, Yifan & Lu, Yikang & Shen, Chen & Shi, Lei, 2022. "Reinforcement learning explains various conditional cooperation," Applied Mathematics and Computation, Elsevier, vol. 427(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Xie, Kai & Szolnoki, Attila, 2026. "Reinforcement learning in evolutionary game theory: A brief review of recent developments," Applied Mathematics and Computation, Elsevier, vol. 510(C).
    2. Zhang, Yongqiang & Zheng, Zehao & Zhang, Xiaoming & Ma, Jinlong, 2025. "A layered strategy updating mechanism for spatial public goods game with punishment," Chaos, Solitons & Fractals, Elsevier, vol. 201(P2).
    3. Kang, Hongwei & Jiang, Chao & Shen, Yong & Sun, Xingping & Chen, Qingyi, 2025. "Neighbor-aware reinforcement learning fosters cooperation in spatial public goods games," Chaos, Solitons & Fractals, Elsevier, vol. 199(P3).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yang, Yujin & Zhao, Dawei & Wang, Juan, 2025. "Evolution of cooperation in spatial public goods games driven by reinforcement learning and environmental feedback," Chaos, Solitons & Fractals, Elsevier, vol. 199(P1).
    2. Zhang, Yongqiang & Zheng, Zehao & Zhang, Xiaoming & Ma, Jinlong, 2025. "A layered strategy updating mechanism for spatial public goods game with punishment," Chaos, Solitons & Fractals, Elsevier, vol. 201(P2).
    3. Huang, Yijie, 2025. "The evolution of cooperation in multi-games with reinforcement learning," Chaos, Solitons & Fractals, Elsevier, vol. 201(P2).
    4. Li, Jiaying & Lv, Shaojie & Zhao, Changheng, 2025. "Public goods games with environmental feedbacks in well-mixed and structured populations," Chaos, Solitons & Fractals, Elsevier, vol. 201(P2).
    5. Xie, Kai & Szolnoki, Attila, 2026. "Reinforcement learning in evolutionary game theory: A brief review of recent developments," Applied Mathematics and Computation, Elsevier, vol. 510(C).
    6. Zhou, Meng & Yang, Yanlong & Xiang, Shuwen, 2025. "Effect of community learning mechanism on cooperation in conflict societies," Chaos, Solitons & Fractals, Elsevier, vol. 192(C).
    7. Yan, Fang & Hou, Xiaorong & Tian, Tingting & Chen, Xiaojie, 2023. "Nonlinear model reference adaptive control approach for governance of the commons in a feedback-evolving game," Chaos, Solitons & Fractals, Elsevier, vol. 174(C).
    8. Luigi Butera & Philip Grossman & Daniel Houser & John List & Marie-Claire Villeval, 2020. "A New Mechanism to Alleviate the Crises of Confidence in Science - With an Application to the Public Goods Game," Artefactual Field Experiments 00684, The Field Experiments Website.
    9. Christiane Reif & Dirk Rübbelke & Andreas Löschel, 2017. "Improving Voluntary Public Good Provision Through a Non-governmental, Endogenous Matching Mechanism: Experimental Evidence," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 67(3), pages 559-589, July.
    10. Huang, Jiaquan & Zhu, Yuying & Xia, Chengyi & Tanimoto, Jun, 2024. "The effect of nonlinear environmental feedback on the outcomes of evolutionary dynamics," Applied Mathematics and Computation, Elsevier, vol. 483(C).
    11. Li, Minlan & Wang, Chao & Han, Yanyan & Wang, Si-Yi & Wang, Ruiwu, 2025. "Environmental feedback promotes cooperation in a spatial prisoner's dilemma game with preferential selection," Applied Mathematics and Computation, Elsevier, vol. 495(C).
    12. Mathieu Lefebvre & Anne Stenger, 2020. "Short- & long-term effects of monetary and non-monetary incentives to cooperate in public good games: An experiment," PLOS ONE, Public Library of Science, vol. 15(1), pages 1-17, January.
    13. Lee, Hsuan-Wei & Weng, Yi-Ning, 2025. "Granular Q-learning adaptation boosts collective welfare in multi-agent Prisoner’s Dilemma," Chaos, Solitons & Fractals, Elsevier, vol. 199(P1).
    14. Luigi Butera & John A. List, 2017. "An Economic Approach to Alleviate the Crises of Confidence in Science: With an Application to the Public Goods Game," NBER Working Papers 23335, National Bureau of Economic Research, Inc.
    15. Krellner, Marcus & Han, The Anh, 2025. "Words are not wind - how public joint commitment and reputation solve the Prisoner's Dilemma," Applied Mathematics and Computation, Elsevier, vol. 506(C).
    16. Maho Nakagawa & Mathieu Lefebvre & Anne Stenger, 2022. "Long-lasting effects of incentives and social preference: A public goods experiment," PLOS ONE, Public Library of Science, vol. 17(8), pages 1-24, August.
    17. Di, Changyan & Zhou, Qingguo & Shen, Jun & Wang, Jinqiang & Zhou, Rui & Wang, Tianyi, 2023. "The coupling effect between the environment and strategies drives the emergence of group cooperation," Chaos, Solitons & Fractals, Elsevier, vol. 176(C).
    18. Brañas-Garza, Pablo & Molis, Elena & Neyse, Levent, 2021. "Exposure to inequality may cause under-provision of public goods: Experimental evidence," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 92(C).
    19. Ananish Chaudhuri, 2018. "Belief Heterogeneity and the Restart Effect in a Public Goods Game," Games, MDPI, vol. 9(4), pages 1-20, November.
    20. Ananish Chaudhuri, 2011. "Sustaining cooperation in laboratory public goods experiments: a selective survey of the literature," Experimental Economics, Springer;Economic Science Association, vol. 14(1), pages 47-83, March.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:chsofr:v:195:y:2025:i:c:s0960077925003091. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Thayer, Thomas R. (email available below). General contact details of provider: https://www.journals.elsevier.com/chaos-solitons-and-fractals .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.