IDEAS home Printed from https://ideas.repec.org/a/eee/chsofr/v203y2026ics096007792501611x.html

Self-confirming Q-learning on unknown networks

Author

Listed:
  • Wang, Xianjia
  • Wang, Linlin
  • Hu, Yaozhong

Abstract

In a large society with millions of agents and non-regular connections, where each agent’s environment is coupled with the actions of other agents, it is then natural to assume that the agents are unaware of the structure of the social network and do not know accurately the environment they are in. This makes them hard to perform complex strategic reasoning due to the incomplete information. To make reasonable decision based on the partial information they can possibly gather they need to react optimally to some subjective conjectures from their observational feedbacks. In this context, this paper explores the self-confirming Q-learning algorithm. This paper defines self-confirming Q-learning equilibria (QSCE), proves the convergence of the self-confirming Q-function, derives the optimal response function for self-confirming Q-learning, and gives the characteristics of QSCE. The analytic results are compared with the results under the offline self-confirming Q-learning process in an example of a six-agent network, and the consistency is verified, which shows the plausibility of self-confirming Q-learning on unknown networks and its equilibrium as proposed in this paper.

Suggested Citation

  • Wang, Xianjia & Wang, Linlin & Hu, Yaozhong, 2026. "Self-confirming Q-learning on unknown networks," Chaos, Solitons & Fractals, Elsevier, vol. 203(C).
    • Handle: RePEc:eee:chsofr:v:203:y:2026:i:c:s096007792501611x
    DOI: 10.1016/j.chaos.2025.117598
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S096007792501611X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.chaos.2025.117598?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Xie, Kai & Szolnoki, Attila, 2025. "Reputation in public goods cooperation under double Q-learning protocol," Chaos, Solitons & Fractals, Elsevier, vol. 196(C).
    2. Fudenberg, Drew & Levine, David K, 1993. "Self-Confirming Equilibrium," Econometrica, Econometric Society, vol. 61(3), pages 523-545, May.
    3. Meng, Dawen & Sun, Lei & Tian, Guoqiang, 2022. "Dynamic mechanism design on social networks," Games and Economic Behavior, Elsevier, vol. 131(C), pages 84-120.
    4. Ozan Candogan, 2022. "Persuasion in Networks: Public Signals and Cores," Operations Research, INFORMS, vol. 70(4), pages 2264-2298, July.
    5. Yann Bramoullé & Rachel Kranton, 2015. "Games Played on Networks," AMSE Working Papers 1530, Aix-Marseille School of Economics, France.
    6. McBride, Michael, 2006. "Imperfect monitoring in communication networks," Journal of Economic Theory, Elsevier, vol. 126(1), pages 97-119, January.
    7. Lukyanov, Georgy & Shamruk, Konstantin & Su, Tong & Wakrim, Ahmed, 2022. "Public communication with externalities," Games and Economic Behavior, Elsevier, vol. 136(C), pages 177-196.
    8. Elliot Lipnowski & Evan Sadler, 2019. "Peer‐Confirming Equilibrium," Econometrica, Econometric Society, vol. 87(2), pages 567-591, March.
    9. Meng, Delong, 2021. "Learning from like-minded people," Games and Economic Behavior, Elsevier, vol. 126(C), pages 231-250.
    10. Lin, Jiaying & Long, Pinduo & Liang, Jinfeng & Dai, Qionglin & Li, Haihong & Yang, Junzhong, 2025. "The coevolution of cooperation: Integrating Q-learning and occasional social interactions in evolutionary games," Chaos, Solitons & Fractals, Elsevier, vol. 194(C).
    11. Zenou, Yves & Bochet, Olivier & Faure, Mathieu & Long, Yan, 2020. "Perceived Competition in Networks," CEPR Discussion Papers 15582, C.E.P.R. Discussion Papers.
    12. Battigalli, Pierpaolo & Panebianco, Fabrizio & Pin, Paolo, 2023. "Learning and selfconfirming equilibria in network games," Journal of Economic Theory, Elsevier, vol. 212(C).
    13. Bramoulle, Yann & Kranton, Rachel, 2007. "Public goods in networks," Journal of Economic Theory, Elsevier, vol. 135(1), pages 478-494, July.
    14. Lefeng Cheng & Xin Wei & Manling Li & Can Tan & Meng Yin & Teng Shen & Tao Zou, 2024. "Integrating Evolutionary Game-Theoretical Methods and Deep Reinforcement Learning for Adaptive Strategy Optimization in User-Side Electricity Markets: A Comprehensive Review," Mathematics, MDPI, vol. 12(20), pages 1-56, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Battigalli, Pierpaolo & Panebianco, Fabrizio & Pin, Paolo, 2023. "Learning and selfconfirming equilibria in network games," Journal of Economic Theory, Elsevier, vol. 212(C).
    2. Olivier Bochet & Mathieu Faure & Yan Long & Yves Zenou, 2025. "Perceived Competition," AMSE Working Papers 2515, Aix-Marseille School of Economics, France.
    3. Bochet, Olivier & Faure, Mathieu & Long, Yan & Zenou, Yves, 2020. "Perceived Competition in Networks," CEPR Discussion Papers 15582, C.E.P.R. Discussion Papers.
    4. Chen, Ying-Ju & Zenou, Yves & Zhou, Junjie, 2022. "The impact of network topology and market structure on pricing," Journal of Economic Theory, Elsevier, vol. 204(C).
    5. Cerreia-Vioglio, Simone & Maccheroni, Fabio & Schmeidler, David, 2022. "Equilibria of nonatomic anonymous games," Games and Economic Behavior, Elsevier, vol. 135(C), pages 110-131.
    6. Foerster, Manuel & Mauleon, Ana & Vannetelbosch, Vincent J., 2021. "Shadow links," Journal of Economic Theory, Elsevier, vol. 197(C).
      • FOERSTER Manuel, & MAULEON Ana, & VANNETELBOSCH Vincent,, 2018. "Shadow links," LIDAM Discussion Papers CORE 2018030, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
      • Foerster, Manuel & Mauleon, Ana & Vannetelbosch, Vincent, 2021. "Shadow links," LIDAM Reprints CORE 3171, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    7. McBride, Michael, 2008. "Position-specific information in social networks: Are you connected?," Mathematical Social Sciences, Elsevier, vol. 56(2), pages 283-295, September.
    8. Dev, Pritha, 2010. "Choosing `Me' and `My Friends': Identity in a Non-Cooperative Network Formation Game with Cost Sharing," MPRA Paper 21631, University Library of Munich, Germany.
    9. Xie, Kai & Szolnoki, Attila, 2026. "Reinforcement learning in evolutionary game theory: A brief review of recent developments," Applied Mathematics and Computation, Elsevier, vol. 510(C).
    10. Walsh, A. M., 2019. "Games on Multi-Layer Networks," Cambridge Working Papers in Economics 1954, Faculty of Economics, University of Cambridge.
    11. Allouch, Nizar, 2017. "The cost of segregation in (social) networks," Games and Economic Behavior, Elsevier, vol. 106(C), pages 329-342.
    12. Parise, Francesca & Ozdaglar, Asuman, 2019. "A variational inequality framework for network games: Existence, uniqueness, convergence and sensitivity analysis," Games and Economic Behavior, Elsevier, vol. 114(C), pages 47-82.
    13. Koessler, Frederic & Pahlke, Marieke, 2025. "Feedback design in games with ambiguity-averse players," Journal of Economic Theory, Elsevier, vol. 225(C).
    14. Pathak, Shraddha & Kulkarni, Ankur A., 2025. "A scalable Bayesian persuasion framework for epidemic containment on heterogeneous networks," Journal of Mathematical Economics, Elsevier, vol. 119(C).
    15. Dev, Pritha, 2010. "Identity and Fragmentation in Networks," MPRA Paper 21632, University Library of Munich, Germany.
    16. Ushchev, Philip & Zenou, Yves, 2020. "Social norms in networks," Journal of Economic Theory, Elsevier, vol. 185(C).
    17. Clyde, Alexander, 2025. "Proxy variables and feedback effects in decision making," Games and Economic Behavior, Elsevier, vol. 153(C), pages 408-429.
    18. Papadimitriou, Christos & Peng, Binghui, 2023. "Public goods games in directed networks," Games and Economic Behavior, Elsevier, vol. 139(C), pages 161-179.
    19. Fudenberg, Drew & Levine, David K., 1999. "Conditional Universal Consistency," Games and Economic Behavior, Elsevier, vol. 29(1-2), pages 104-130, October.
    20. Karan N. Chadha & Ankur A. Kulkarni, 2022. "On independent cliques and linear complementarity problems," Indian Journal of Pure and Applied Mathematics, Springer, vol. 53(4), pages 1036-1057, December.

    More about this item

    Keywords

    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:chsofr:v:203:y:2026:i:c:s096007792501611x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Thayer, Thomas R. (email available below). General contact details of provider: https://www.journals.elsevier.com/chaos-solitons-and-fractals .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.