IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0301141.html
   My bibliography  Save this article

Mesoscale effects of trader learning behaviors in financial markets: A multi-agent reinforcement learning study

Author

Listed:
  • Johann Lussange
  • Stefano Vrizzi
  • Stefano Palminteri
  • Boris Gutkin

Abstract

Recent advances in the field of machine learning have yielded novel research perspectives in behavioural economics and financial markets microstructure studies. In this paper we study the impact of individual trader leaning characteristics on markets using a stock market simulator designed with a multi-agent architecture. Each agent, representing an autonomous investor, trades stocks through reinforcement learning, using a centralized double-auction limit order book. This approach allows us to study the impact of individual trader traits on the whole stock market at the mesoscale in a bottom-up approach. We chose to test three trader trait aspects: agent learning rate increases, herding behaviour and random trading. As hypothesized, we find that larger learning rates significantly increase the number of crashes. We also find that herding behaviour undermines market stability, while random trading tends to preserve it.

Suggested Citation

  • Johann Lussange & Stefano Vrizzi & Stefano Palminteri & Boris Gutkin, 2024. "Mesoscale effects of trader learning behaviors in financial markets: A multi-agent reinforcement learning study," PLOS ONE, Public Library of Science, vol. 19(4), pages 1-40, April.
    • Handle: RePEc:plo:pone00:0301141
    DOI: 10.1371/journal.pone.0301141
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0301141
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0301141&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0301141?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Roberto Mota Navarro & Hern'an Larralde Ridaura, 2016. "A detailed heterogeneous agent model for a single asset financial market with trading via an order book," Papers 1601.00229, arXiv.org, revised Jul 2016.
    2. John Y. Campbell & Sanford J. Grossman & Jiang Wang, 1993. "Trading Volume and Serial Correlation in Stock Returns," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 108(4), pages 905-939.
    3. Stephen J. Hardiman & Nicolas Bercot & Jean-Philippe Bouchaud, 2013. "Critical reflexivity in financial markets: a Hawkes process analysis," Papers 1302.1405, arXiv.org, revised Jun 2013.
    4. Chiarella, Carl & Iori, Giulia, 2009. "The impact of heterogeneous trading rules on the limit order book and order flows," Journal of Economic Dynamics and Control, Elsevier, vol. 33(3), pages 525-537.
    5. Salle, Isabelle & Yildizoglu, Murat & Zumpe, Martin & Sénégas, Marc-Alexandre, 2017. "Coordination through social learning in a general equilibrium model," Journal of Economic Behavior & Organization, Elsevier, vol. 141(C), pages 64-82.
    6. Stefano Palminteri & Mehdi Khamassi & Mateus Joffily & Giorgio Coricelli, 2015. "Contextual modulation of value signals in reward and punishment learning," Nature Communications, Nature, vol. 6(1), pages 1-14, November.
    7. Grossman, Sanford J & Stiglitz, Joseph E, 1980. "On the Impossibility of Informationally Efficient Markets," American Economic Review, American Economic Association, vol. 70(3), pages 393-408, June.
    8. Sylvain Barde, 2015. "Direct calibration and comparison of agent-based herding models of financial markets," Studies in Economics 1507, School of Economics, University of Kent.
    9. Stefano Palminteri & Germain Lefebvre & Emma J. Kilford & Sarah-Jayne Blakemore, 2017. "Confirmation bias in human reinforcement learning: Evidence from counterfactual feedback processing," Post-Print hal-04134742, HAL.
    10. Elena Green & Daniel M. Heffernan, 2019. "An Agent-Based Model to Explain the Emergence of Stylised Facts in Log Returns," Papers 1901.05053, arXiv.org.
    11. Salle, Isabelle & Yildizoglu, Murat & Zumpe, Martin & Sénégas, Marc-Alexandre, 2017. "Coordination through social learning in a general equilibrium model," Journal of Economic Behavior & Organization, Elsevier, vol. 141(C), pages 64-82.
    12. Cont, Rama & Bouchaud, Jean-Philipe, 2000. "Herd Behavior And Aggregate Fluctuations In Financial Markets," Macroeconomic Dynamics, Cambridge University Press, vol. 4(2), pages 170-196, June.
    13. Justin Sirignano & Rama Cont, 2019. "Universal features of price formation in financial markets: perspectives from deep learning," Quantitative Finance, Taylor & Francis Journals, vol. 19(9), pages 1449-1459, September.
    14. Charles Cao & Yong Chen & William N. Goetzmann & Bing Liang, 2018. "Hedge Funds and Stock Price Formation," Financial Analysts Journal, Taylor & Francis Journals, vol. 74(3), pages 54-68, July.
    15. Alessio Emanuele Biondo, 2019. "Order book modeling and financial stability," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 14(3), pages 469-489, September.
    16. Aleksandras Vytautas Rutkauskas & Tomas Ramanauskas, 2009. "Building an artificial stock market populated by reinforcement‐learning agents," Journal of Business Economics and Management, Taylor & Francis Journals, vol. 10(4), pages 329-341, September.
    17. Marcus Cordi & Damien Challet & Ioane Muni Toke, 2017. "Testing the causality of Hawkes processes with time reversal," Papers 1709.08516, arXiv.org.
    18. Stefano Palminteri & Mehdi Khamassi & Mateus Joffily & Giorgio Coricelli, 2015. "Contextual modulation of value signals in reward and punishment learning," Post-Print halshs-01236045, HAL.
    19. Westerhoff Frank H., 2008. "The Use of Agent-Based Financial Market Models to Test the Effectiveness of Regulatory Policies," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 228(2-3), pages 195-227, April.
    20. Tomas Ramanauskas & Aleksandras Vytautas Rutkauskas, 2009. "Building an Artificial Stock Market Populated by Reinforcement-Learning Agents," Bank of Lithuania Working Paper Series 6, Bank of Lithuania.
    21. Weibing Huang & Charles-Albert Lehalle & Mathieu Rosenbaum, 2015. "Simulating and Analyzing Order Book Data: The Queue-Reactive Model," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(509), pages 107-122, March.
    22. Gualdi, Stanislao & Tarzia, Marco & Zamponi, Francesco & Bouchaud, Jean-Philippe, 2015. "Tipping points in macroeconomic agent-based models," Journal of Economic Dynamics and Control, Elsevier, vol. 50(C), pages 29-61.
    23. Gode, Dhananjay K & Sunder, Shyam, 1993. "Allocative Efficiency of Markets with Zero-Intelligence Traders: Market as a Partial Substitute for Individual Rationality," Journal of Political Economy, University of Chicago Press, vol. 101(1), pages 119-137, February.
    24. Orlean, Andre, 1995. "Bayesian interactions and collective dynamics of opinion: Herd behavior and mimetic contagion," Journal of Economic Behavior & Organization, Elsevier, vol. 28(2), pages 257-274, October.
    25. Pramod Kumar Naik & Rangan Gupta & Puja Padhi, 2018. "The Relationship Between Stock Market Volatility And Trading Volume: Evidence From South Africa," Journal of Developing Areas, Tennessee State University, College of Business, vol. 52(1), pages 99-114, January-M.
    26. Stephen Hardiman & Nicolas Bercot & Jean-Philippe Bouchaud, 2013. "Critical reflexivity in financial markets: a Hawkes process analysis," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 86(10), pages 1-9, October.
    27. Anna Dodonova & Yuri Khoroshilov, 2018. "Private information in futures markets: An experimental study," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 39(1), pages 65-70, January.
    28. R. Cont, 2001. "Empirical properties of asset returns: stylized facts and statistical issues," Quantitative Finance, Taylor & Francis Journals, vol. 1(2), pages 223-236.
    29. Marco Corazza & Francesco Bertoluzzo, 2014. "Q-Learning-based financial trading systems with applications," Working Papers 2014:15, Department of Economics, University of Venice "Ca' Foscari".
    30. Stefano Palminteri & Germain Lefebvre & Emma J Kilford & Sarah-Jayne Blakemore, 2017. "Confirmation bias in human reinforcement learning: Evidence from counterfactual feedback processing," PLOS Computational Biology, Public Library of Science, vol. 13(8), pages 1-22, August.
    31. Fama, Eugene F, 1970. "Efficient Capital Markets: A Review of Theory and Empirical Work," Journal of Finance, American Finance Association, vol. 25(2), pages 383-417, May.
    32. Yagna Patel, 2018. "Optimizing Market Making using Multi-Agent Reinforcement Learning," Papers 1812.10252, arXiv.org.
    33. Sumitra Ganesh & Nelson Vadori & Mengda Xu & Hua Zheng & Prashant Reddy & Manuela Veloso, 2019. "Reinforcement Learning for Market Making in a Multi-agent Dealer Market," Papers 1911.05892, arXiv.org.
    34. Jonathan Lewellen, 2002. "Momentum and Autocorrelation in Stock Returns," The Review of Financial Studies, Society for Financial Studies, vol. 15(2), pages 533-564, March.
    35. Fischer, Thomas G., 2018. "Reinforcement learning in financial markets - a survey," FAU Discussion Papers in Economics 12/2018, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    36. Germain Lefebvre & Maël Lebreton & Florent Meyniel & Sacha Bourgeois-Gironde & Stefano Palminteri, 2017. "Behavioural and neural characterization of optimistic reinforcement learning," Nature Human Behaviour, Nature, vol. 1(4), pages 1-9, April.
    37. De Vries, C.G. & Leuven, K.U., 1994. "Stylized Facts of Nominal Exchange Rate Returns," Papers 94-002, Purdue University, Krannert School of Management - Center for International Business Education and Research (CIBER).
    38. I. Momennejad & E. M. Russek & J. H. Cheong & M. M. Botvinick & N. D. Daw & S. J. Gershman, 2017. "The successor representation in human reinforcement learning," Nature Human Behaviour, Nature, vol. 1(9), pages 680-692, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Johann Lussange & Stefano Vrizzi & Stefano Palminteri & Boris Gutkin, 2024. "Mesoscale effects of trader learning behaviors in financial markets: A multi-agent reinforcement learning study," Post-Print hal-04790290, HAL.
    2. Johann Lussange & Stefano Vrizzi & Sacha Bourgeois-Gironde & Stefano Palminteri & Boris Gutkin, 2023. "Stock Price Formation: Precepts from a Multi-Agent Reinforcement Learning Model," Computational Economics, Springer;Society for Computational Economics, vol. 61(4), pages 1523-1544, April.
    3. Johann Lussange & Ivan Lazarevich & Sacha Bourgeois-Gironde & Stefano Palminteri & Boris Gutkin, 2021. "Modelling Stock Markets by Multi-agent Reinforcement Learning," Computational Economics, Springer;Society for Computational Economics, vol. 57(1), pages 113-147, January.
    4. Johann Lussange & Stefano Vrizzi & Stefano Palminteri & Boris Gutkin, 2024. "Modelling crypto markets by multi-agent reinforcement learning," Papers 2402.10803, arXiv.org.
    5. Johann Lussange & Boris Gutkin, 2023. "Order book regulatory impact on stock market quality: a multi-agent reinforcement learning perspective," Papers 2302.04184, arXiv.org.
    6. Luis Goncalves de Faria, 2022. "An Agent-Based Model With Realistic Financial Time Series: A Method for Agent-Based Models Validation," Papers 2206.09772, arXiv.org.
    7. Jovanovic, Franck & Schinckus, Christophe, 2017. "Econophysics and Financial Economics: An Emerging Dialogue," OUP Catalogue, Oxford University Press, number 9780190205034, Decembrie.
    8. Torsten Trimborn & Philipp Otte & Simon Cramer & Maximilian Beikirch & Emma Pabich & Martin Frank, 2020. "SABCEMM: A Simulator for Agent-Based Computational Economic Market Models," Computational Economics, Springer;Society for Computational Economics, vol. 55(2), pages 707-744, February.
    9. Martin D. Gould & Mason A. Porter & Stacy Williams & Mark McDonald & Daniel J. Fenn & Sam D. Howison, 2010. "Limit Order Books," Papers 1012.0349, arXiv.org, revised Apr 2013.
    10. Ben Hambly & Renyuan Xu & Huining Yang, 2021. "Recent Advances in Reinforcement Learning in Finance," Papers 2112.04553, arXiv.org, revised Feb 2023.
    11. Miko{l}aj Bi'nkowski & Charles-Albert Lehalle, 2018. "Endogeneous Dynamics of Intraday Liquidity," Papers 1811.03766, arXiv.org.
    12. Arvid Oskar Ivar Hoffmann & Wander Jager & J. H. Von Eije, 2007. "Social Simulation of Stock Markets: Taking It to the Next Level," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 10(2), pages 1-7.
    13. Alessio Emanuele Biondo, 2019. "Order book modeling and financial stability," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 14(3), pages 469-489, September.
    14. Didier SORNETTE, 2014. "Physics and Financial Economics (1776-2014): Puzzles, Ising and Agent-Based Models," Swiss Finance Institute Research Paper Series 14-25, Swiss Finance Institute.
    15. Ashok Chanabasangouda Patil & Shailesh Rastogi, 2019. "Time-Varying Price–Volume Relationship and Adaptive Market Efficiency: A Survey of the Empirical Literature," JRFM, MDPI, vol. 12(2), pages 1-18, June.
    16. Noemi Schmitt & Ivonne Schwartz & Frank Westerhoff, 2022. "Heterogeneous speculators and stock market dynamics: a simple agent-based computational model," The European Journal of Finance, Taylor & Francis Journals, vol. 28(13-15), pages 1263-1282, October.
    17. Pankaj Kumar, 2021. "Deep Hawkes Process for High-Frequency Market Making," Papers 2109.15110, arXiv.org.
    18. Adri'an Carro & Ra'ul Toral & Maxi San Miguel, 2015. "Markets, herding and response to external information," Papers 1506.03708, arXiv.org, revised Jun 2015.
    19. D. Sornette, 2014. "Physics and Financial Economics (1776-2014): Puzzles, Ising and Agent-Based models," Papers 1404.0243, arXiv.org.
    20. Adrián Carro & Raúl Toral & Maxi San Miguel, 2015. "Markets, Herding and Response to External Information," PLOS ONE, Public Library of Science, vol. 10(7), pages 1-28, July.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0301141. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.