IDEAS home Printed from https://ideas.repec.org/a/eee/apmaco/v413y2022ics0096300321007426.html
   My bibliography  Save this article

Empty nodes affect conditional cooperation under reinforcement learning

Author

Listed:
  • Jia, Danyang
  • Li, Tong
  • Zhao, Yang
  • Zhang, Xiaoqin
  • Wang, Zhen

Abstract

In social dilemmas, individual behavior generally follows the characteristics of conditional cooperation and emotional conditional cooperation. However, it is hard to adequately explain the behavior patterns of conditional cooperation with the evolutionary game theory. This paper introduces expectation-based reinforcement learning methods in the public goods game to investigate and account for the behavior patterns. Instead of letting individuals occupy the entire network as previous studies have done, we focus on studying individual behavior patterns on a network with empty nodes. The results under total population density show the effectivity of our model as they are consistent with those of the previous studies, that is, individuals’ behavior exhibits conditional cooperation and its variant moody conditional cooperation. However, in the network with empty nodes, conditional cooperation shows opposite trends. We finally demonstrate that an appropriate population density can facilitate the maintenance and development of cooperation.

Suggested Citation

  • Jia, Danyang & Li, Tong & Zhao, Yang & Zhang, Xiaoqin & Wang, Zhen, 2022. "Empty nodes affect conditional cooperation under reinforcement learning," Applied Mathematics and Computation, Elsevier, vol. 413(C).
    • Handle: RePEc:eee:apmaco:v:413:y:2022:i:c:s0096300321007426
    DOI: 10.1016/j.amc.2021.126658
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0096300321007426
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.amc.2021.126658?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Schlag, Karl H., 1999. "Which one should I imitate?," Journal of Mathematical Economics, Elsevier, vol. 31(4), pages 493-522, May.
    2. Christian Hilbe & Krishnendu Chatterjee & Martin A. Nowak, 2018. "Publisher Correction: Partners and rivals in direct reciprocity," Nature Human Behaviour, Nature, vol. 2(7), pages 523-523, July.
    3. Christian Hilbe & Krishnendu Chatterjee & Martin A. Nowak, 2018. "Partners and rivals in direct reciprocity," Nature Human Behaviour, Nature, vol. 2(7), pages 469-477, July.
    4. Fischbacher, Urs & Gachter, Simon & Fehr, Ernst, 2001. "Are people conditionally cooperative? Evidence from a public goods experiment," Economics Letters, Elsevier, vol. 71(3), pages 397-404, June.
    5. Claudia Keser & Frans Van Winden, 2000. "Conditional Cooperation and Voluntary Contributions to Public Goods," Scandinavian Journal of Economics, Wiley Blackwell, vol. 102(1), pages 23-39, March.
    6. Takahiro Ezaki & Yutaka Horita & Masanori Takezawa & Naoki Masuda, 2016. "Reinforcement Learning Explains Conditional Cooperation and Its Moody Cousin," PLOS Computational Biology, Public Library of Science, vol. 12(7), pages 1-13, July.
    7. Jelena Grujić & Torsten Röhl & Dirk Semmann & Manfred Milinski & Arne Traulsen, 2012. "Consistent Strategy Updating in Spatial and Non-Spatial Behavioral Experiments Does Not Promote Cooperation in Social Networks," PLOS ONE, Public Library of Science, vol. 7(11), pages 1-8, November.
    8. Zhen Wang & Marko Jusup & Lei Shi & Joung-Hun Lee & Yoh Iwasa & Stefano Boccaletti, 2018. "Exploiting a cognitive bias promotes cooperation in social dilemma experiments," Nature Communications, Nature, vol. 9(1), pages 1-7, December.
    9. Fernando P. Santos & Francisco C. Santos & Jorge M. Pacheco, 2018. "Social norm complexity and past reputations in the evolution of cooperation," Nature, Nature, vol. 555(7695), pages 242-245, March.
    10. Schlag, Karl H., 1998. "Why Imitate, and If So, How?, : A Boundedly Rational Approach to Multi-armed Bandits," Journal of Economic Theory, Elsevier, vol. 78(1), pages 130-156, January.
    11. Ernst Fehr & Urs Fischbacher, 2004. "Social norms and human cooperation," Macroeconomics 0409026, University Library of Munich, Germany.
    12. Hisashi Ohtsuki & Christoph Hauert & Erez Lieberman & Martin A. Nowak, 2006. "A simple rule for the evolution of cooperation on graphs and social networks," Nature, Nature, vol. 441(7092), pages 502-505, May.
    13. Segismundo S. Izquierdo & Luis R. Izquierdo & Nicholas M. Gotts, 2008. "Reinforcement Learning Dynamics in Social Dilemmas," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 11(2), pages 1-1.
    14. Izquierdo, Luis R. & Izquierdo, Segismundo S. & Gotts, Nicholas M. & Polhill, J. Gary, 2007. "Transient and asymptotic dynamics of reinforcement learning in games," Games and Economic Behavior, Elsevier, vol. 61(2), pages 259-276, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Du, Chunpeng & Guo, Keyu & Lu, Yikang & Jin, Haoyu & Shi, Lei, 2023. "Aspiration driven exit-option resolves social dilemmas in the network," Applied Mathematics and Computation, Elsevier, vol. 438(C).
    2. You, Tao & Yang, Haochun & Wang, Jian & Zhang, Peng & Chen, Jinchao & Zhang, Ying, 2023. "Cooperative behavior under the influence of multiple experienced guiders in Prisoner’s dilemma game," Applied Mathematics and Computation, Elsevier, vol. 458(C).
    3. Meng, Xueyu & Lin, Jianhong & Fan, Yufei & Gao, Fujuan & Fenoaltea, Enrico Maria & Cai, Zhiqiang & Si, Shubin, 2023. "Coupled disease-vaccination behavior dynamic analysis and its application in COVID-19 pandemic," Chaos, Solitons & Fractals, Elsevier, vol. 169(C).
    4. Xiaopeng Li & Zhonglin Wang & Jiuqiang Liu & Guihai Yu, 2023. "The Sense of Cooperation on Interdependent Networks Inspired by Influence-Based Self-Organization," Mathematics, MDPI, vol. 11(4), pages 1-16, February.
    5. Bai, Pengzhou & Qiang, Bingzhuang & Zou, Kuan & Huang, Changwei, 2024. "Preferential selection based on adaptive attractiveness induce by reinforcement learning promotes cooperation," Chaos, Solitons & Fractals, Elsevier, vol. 180(C).
    6. Xu, Yan & Feng, Meiling & Zhu, Yuying & Xia, Chengyi, 2022. "Multi-player snowdrift game on scale-free simplicial complexes," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 604(C).
    7. You, Tao & Zhang, Hailun & Zhang, Ying & Li, Qing & Zhang, Peng & Yang, Mei, 2022. "The influence of experienced guider on cooperative behavior in the Prisoner’s dilemma game," Applied Mathematics and Computation, Elsevier, vol. 426(C).
    8. Zheng, Guozhong & Zhang, Jiqiang & Deng, Shengfeng & Cai, Weiran & Chen, Li, 2024. "Evolution of cooperation in the public goods game with Q-learning," Chaos, Solitons & Fractals, Elsevier, vol. 188(C).
    9. Zhang, Huizhen & An, Tianbo & Yan, Pingping & Hu, Kaipeng & An, Jinjin & Shi, Lijuan & Zhao, Jian & Wang, Jingrui, 2024. "Exploring cooperative evolution with tunable payoff’s loners using reinforcement learning," Chaos, Solitons & Fractals, Elsevier, vol. 178(C).
    10. Wang, Xianjia & Yang, Zhipeng & Liu, Yanli & Chen, Guici, 2023. "A reinforcement learning-based strategy updating model for the cooperative evolution," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 618(C).
    11. Flores, Lucas S. & Amaral, Marco A. & Vainstein, Mendeli H. & Fernandes, Heitor C.M., 2022. "Cooperation in regular lattices," Chaos, Solitons & Fractals, Elsevier, vol. 164(C).
    12. Ding, Zhen-Wei & Zheng, Guo-Zhong & Cai, Chao-Ran & Cai, Wei-Ran & Chen, Li & Zhang, Ji-Qiang & Wang, Xu-Ming, 2023. "Emergence of cooperation in two-agent repeated games with reinforcement learning," Chaos, Solitons & Fractals, Elsevier, vol. 175(P1).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Takahiro Ezaki & Yutaka Horita & Masanori Takezawa & Naoki Masuda, 2016. "Reinforcement Learning Explains Conditional Cooperation and Its Moody Cousin," PLOS Computational Biology, Public Library of Science, vol. 12(7), pages 1-13, July.
    2. Yali Dong & Cong Li & Yi Tao & Boyu Zhang, 2015. "Evolution of Conformity in Social Dilemmas," PLOS ONE, Public Library of Science, vol. 10(9), pages 1-12, September.
    3. Christian Hilbe & Maria Kleshnina & Kateřina Staňková, 2023. "Evolutionary Games and Applications: Fifty Years of ‘The Logic of Animal Conflict’," Dynamic Games and Applications, Springer, vol. 13(4), pages 1035-1048, December.
    4. repec:grz:wpsses:2021-02 is not listed on IDEAS
    5. Takahiro Ezaki & Naoki Masuda, 2017. "Reinforcement learning account of network reciprocity," PLOS ONE, Public Library of Science, vol. 12(12), pages 1-8, December.
    6. Laura Schmid & Farbod Ekbatani & Christian Hilbe & Krishnendu Chatterjee, 2023. "Quantitative assessment can stabilize indirect reciprocity under imperfect information," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    7. Wang, Xianjia & Yang, Zhipeng & Liu, Yanli & Chen, Guici, 2023. "A reinforcement learning-based strategy updating model for the cooperative evolution," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 618(C).
    8. Shirsendu Podder & Simone Righi, 2023. "Complexity of Behavioural Strategies and Cooperation in the Optional Public Goods Game," Dynamic Games and Applications, Springer, vol. 13(4), pages 1219-1235, December.
    9. Kölle, Felix & Quercia, Simone, 2021. "The influence of empirical and normative expectations on cooperation," Journal of Economic Behavior & Organization, Elsevier, vol. 190(C), pages 691-703.
    10. Reuben, Ernesto & Riedl, Arno, 2013. "Enforcement of contribution norms in public good games with heterogeneous populations," Games and Economic Behavior, Elsevier, vol. 77(1), pages 122-137.
    11. Zhou, Zhizhuo & Rong, Zhihai & Yang, Wen & Wu, Zhi-Xi, 2024. "Coevolution of extortion strategies with mixed imitation and aspiration learning dynamics in spatial Prisoner’s Dilemma game," Chaos, Solitons & Fractals, Elsevier, vol. 188(C).
    12. Zoe Van der Hoven & Martine Visser & Kerri Brick, 2012. "Contribution Norms in Heterogeneous Groups: A Climate Change Framing," SALDRU Working Papers 77, Southern Africa Labour and Development Research Unit, University of Cape Town.
    13. Choi, S. & Goyal, S. & Guo, F. & Moisan, F., 2024. "Experimental Evidence on the Relation Between Network Centrality and Individual Choice," Janeway Institute Working Papers 2401, Faculty of Economics, University of Cambridge.
    14. Geng, Yini & Liu, Yifan & Lu, Yikang & Shen, Chen & Shi, Lei, 2022. "Reinforcement learning explains various conditional cooperation," Applied Mathematics and Computation, Elsevier, vol. 427(C).
    15. Tobias Heldt, 2005. "Conditional cooperation in the field: Cross-country skiers' behavior in sweden," Natural Field Experiments 00268, The Field Experiments Website.
    16. Mullins, Daniel A. & Whitehouse, Harvey & Atkinson, Quentin D., 2013. "The role of writing and recordkeeping in the cultural evolution of human cooperation," Journal of Economic Behavior & Organization, Elsevier, vol. 90(S), pages 141-151.
    17. Malte Baader & Simon Gaechter & Kyeongtae Lee & Martin Sefton, 2022. "Social preferences and the variability of conditional cooperation," Discussion Papers 2022-13, The Centre for Decision Research and Experimental Economics, School of Economics, University of Nottingham.
    18. Han, Xu & Zhao, Xiaowei & Xia, Haoxiang, 2022. "Hybrid learning promotes cooperation in the spatial prisoner’s dilemma game," Chaos, Solitons & Fractals, Elsevier, vol. 164(C).
    19. Tsakas Nikolas, 2014. "Imitating the Most Successful Neighbor in Social Networks," Review of Network Economics, De Gruyter, vol. 12(4), pages 403-435, February.
    20. Claudio J. Tessone & Angel Sanchez & Frank Schweitzer, "undated". "Diversity-induced resonance in the response to social norms," Working Papers ETH-RC-12-017, ETH Zurich, Chair of Systems Design.
    21. Quan, Ji & Chen, Xinyue & Wang, Xianjia, 2024. "Repeated prisoner's dilemma games in multi-player structured populations with crosstalk," Applied Mathematics and Computation, Elsevier, vol. 473(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:apmaco:v:413:y:2022:i:c:s0096300321007426. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/applied-mathematics-and-computation .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.