IDEAS home Printed from https://ideas.repec.org/a/eee/phsmap/v388y2009i9p1849-1856.html
   My bibliography  Save this article

Statistical mechanics approach to a reinforcement learning model with memory

Author

Listed:
  • Lipowski, Adam
  • Gontarek, Krzysztof
  • Ausloos, Marcel

Abstract

We introduce a two-player model of reinforcement learning with memory. Past actions of an iterated game are stored in a memory and used to determine player’s next action. To examine the behaviour of the model some approximate methods are used and confronted against numerical simulations and exact master equation. When the length of memory of players increases to infinity the model undergoes an absorbing-state phase transition. Performance of examined strategies is checked in the prisoner’ dilemma game. It turns out that it is advantageous to have a large memory in symmetric games, but it is better to have a short memory in asymmetric ones.

Suggested Citation

  • Lipowski, Adam & Gontarek, Krzysztof & Ausloos, Marcel, 2009. "Statistical mechanics approach to a reinforcement learning model with memory," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 388(9), pages 1849-1856.
    • Handle: RePEc:eee:phsmap:v:388:y:2009:i:9:p:1849-1856
    DOI: 10.1016/j.physa.2009.01.028
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0378437109000831
    Download Restriction: Full text for ScienceDirect subscribers only. Journal offers the option of making the article available online on Science direct for a fee of $3,000
    File URL: https://libkey.io/10.1016/j.physa.2009.01.028?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Drew Fudenberg & Jean Tirole, 1991. "Game Theory," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262061414, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Liangliang Chang & Zhipeng Zhang & Chengyi Xia, 2023. "Impact of Decision Feedback on Networked Evolutionary Game with Delays in Control Channel," Dynamic Games and Applications, Springer, vol. 13(3), pages 783-800, September.
    2. Liu, Tianhao, 2021. "A study on day-of-week effect of submission: Based on the data of JSFST," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 563(C).
    3. Ausloos, Marcel & Nedic, Olgica & Dekanski, Aleksandar, 2016. "Day of the week effect in paper submission/acceptance/rejection to/in/by peer review journals," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 456(C), pages 197-203.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Janvier D. Nkurunziza, 2005. "Reputation and Credit without Collateral in Africa`s Formal Banking," Economics Series Working Papers WPS/2005-02, University of Oxford, Department of Economics.
    2. Kessing, Sebastian G. & Konrad, Kai A. & Kotsogiannis, Christos, 2006. "Federal tax autonomy and the limits of cooperation," Journal of Urban Economics, Elsevier, vol. 59(2), pages 317-329, March.
    3. Sandy Fréret & Denis Maguain, 2017. "The effects of agglomeration on tax competition: evidence from a two-regime spatial panel model on French data," International Tax and Public Finance, Springer;International Institute of Public Finance, vol. 24(6), pages 1100-1140, December.
    4. , & ,, 2015. "A folk theorem for stochastic games with infrequent state changes," Theoretical Economics, Econometric Society, vol. 10(1), January.
    5. Carlo Rosa & Giovanni Verga, 2006. "The Impact of Central Bank Announcements on Asset Prices in Real Time: Testing the Efficiency of the Euribor Futures Market," CEP Discussion Papers dp0764, Centre for Economic Performance, LSE.
    6. Marco Bassetto, 2002. "A Game-Theoretic View of the Fiscal Theory of the Price Level," Econometrica, Econometric Society, vol. 70(6), pages 2167-2195, November.
    7. Casey Rothschild & Florian Scheuer, 2014. "A Theory of Income Taxation under Multidimensional Skill Heterogeneity," NBER Working Papers 19822, National Bureau of Economic Research, Inc.
    8. Arthur Schram & Boris Van Leeuwen & Theo Offerman, 2013. "Superstars Need Social Benefits: An Experiment on Network Formation," Working Papers 1306, Departament Empresa, Universitat Autònoma de Barcelona, revised Jul 2013.
    9. Feltenstein, Andrew & Lagunoff, Roger, 2005. "International versus domestic auditing of bank solvency," Journal of International Economics, Elsevier, vol. 67(1), pages 73-96, September.
    10. Patrick W. Schmitz, 2006. "Book Review," Journal of Institutional and Theoretical Economics (JITE), Mohr Siebeck, Tübingen, vol. 162(3), pages 535-542, September.
    11. Strausz, Roland, 2006. "Deterministic versus stochastic mechanisms in principal-agent models," Journal of Economic Theory, Elsevier, vol. 128(1), pages 306-314, May.
    12. Olken, Benjamin A., 2009. "Corruption perceptions vs. corruption reality," Journal of Public Economics, Elsevier, vol. 93(7-8), pages 950-964, August.
    13. Bustamante, Maria Cecilia, 2011. "Strategic investment, industry concentration and the cross section of returns," LSE Research Online Documents on Economics 37454, London School of Economics and Political Science, LSE Library.
    14. Celik, Gorkem, 2006. "Mechanism design with weaker incentive compatibility constraints," Games and Economic Behavior, Elsevier, vol. 56(1), pages 37-44, July.
    15. Régis Chenavaz & Corina Paraschiv & Gabriel Turinici, 2017. "Dynamic Pricing of New Products in Competitive Markets: A Mean-Field Game Approach," Working Papers hal-01592958, HAL.
    16. Dirk Bergemann & Stephen Morris, 2019. "Information Design: A Unified Perspective," Journal of Economic Literature, American Economic Association, vol. 57(1), pages 44-95, March.
    17. Kranich, Laurence, 1997. "Equalizing opportunities through public education when innate abilities are unobservable," UC3M Working papers. Economics 7216, Universidad Carlos III de Madrid. Departamento de Economía.
    18. Christoph Engel, 2006. "The Difficult Reception of Rigorous Descriptive Social Science in the Law," Discussion Paper Series of the Max Planck Institute for Research on Collective Goods 2006_1, Max Planck Institute for Research on Collective Goods.
    19. Boone, J., 2003. "Optimal Competition : A Benchmark for Competition Policy," Discussion Paper 2003-3, Tilburg University, Center for Economic Research.
    20. repec:dau:papers:123456789/6818 is not listed on IDEAS
    21. Cantillo, Miguel & Wright, Julian, 2000. "How Do Firms Choose Their Lenders? An Empirical Investigation," The Review of Financial Studies, Society for Financial Studies, vol. 13(1), pages 155-189.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:phsmap:v:388:y:2009:i:9:p:1849-1856. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/physica-a-statistical-mechpplications/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.