IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2403.19781.html
   My bibliography  Save this paper

Reinforcement Learning in Agent-Based Market Simulation: Unveiling Realistic Stylized Facts and Behavior

Author

Listed:
  • Zhiyuan Yao
  • Zheng Li
  • Matthew Thomas
  • Ionut Florescu

Abstract

Investors and regulators can greatly benefit from a realistic market simulator that enables them to anticipate the consequences of their decisions in real markets. However, traditional rule-based market simulators often fall short in accurately capturing the dynamic behavior of market participants, particularly in response to external market impact events or changes in the behavior of other participants. In this study, we explore an agent-based simulation framework employing reinforcement learning (RL) agents. We present the implementation details of these RL agents and demonstrate that the simulated market exhibits realistic stylized facts observed in real-world markets. Furthermore, we investigate the behavior of RL agents when confronted with external market impacts, such as a flash crash. Our findings shed light on the effectiveness and adaptability of RL-based agents within the simulation, offering insights into their response to significant market events.

Suggested Citation

  • Zhiyuan Yao & Zheng Li & Matthew Thomas & Ionut Florescu, 2024. "Reinforcement Learning in Agent-Based Market Simulation: Unveiling Realistic Stylized Facts and Behavior," Papers 2403.19781, arXiv.org.
    • Handle: RePEc:arx:papers:2403.19781
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2403.19781
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Johann Lussange & Ivan Lazarevich & Sacha Bourgeois-Gironde & Stefano Palminteri & Boris Gutkin, 2021. "Modelling Stock Markets by Multi-agent Reinforcement Learning," Computational Economics, Springer;Society for Computational Economics, vol. 57(1), pages 113-147, January.
    2. Rama Cont & Sasha Stoikov & Rishi Talreja, 2010. "A Stochastic Model for Order Book Dynamics," Operations Research, INFORMS, vol. 58(3), pages 549-563, June.
    3. Eric Smith & J Doyne Farmer & Laszlo Gillemot & Supriya Krishnamurthy, 2003. "Statistical theory of the continuous double auction," Quantitative Finance, Taylor & Francis Journals, vol. 3(6), pages 481-514.
    4. R. Cont, 2001. "Empirical properties of asset returns: stylized facts and statistical issues," Quantitative Finance, Taylor & Francis Journals, vol. 1(2), pages 223-236.
    5. Ioanid Rosu, 2009. "A Dynamic Model of the Limit Order Book," The Review of Financial Studies, Society for Financial Studies, vol. 22(11), pages 4601-4641, November.
    6. Ciamac C. Moallemi & Muye Wang, 2022. "A reinforcement learning approach to optimal execution," Quantitative Finance, Taylor & Francis Journals, vol. 22(6), pages 1051-1069, June.
    7. Copeland, Thomas E & Galai, Dan, 1983. "Information Effects on the Bid-Ask Spread," Journal of Finance, American Finance Association, vol. 38(5), pages 1457-1469, December.
    8. Glosten, Lawrence R. & Milgrom, Paul R., 1985. "Bid, ask and transaction prices in a specialist market with heterogeneously informed traders," Journal of Financial Economics, Elsevier, vol. 14(1), pages 71-100, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Martin D. Gould & Mason A. Porter & Stacy Williams & Mark McDonald & Daniel J. Fenn & Sam D. Howison, 2010. "Limit Order Books," Papers 1012.0349, arXiv.org, revised Apr 2013.
    2. Martin D. Gould & Mason A. Porter & Stacy Williams & Mark McDonald & Daniel J. Fenn & Sam D. Howison, 2013. "Limit order books," Quantitative Finance, Taylor & Francis Journals, vol. 13(11), pages 1709-1742, November.
    3. Alessio Emanuele Biondo, 2019. "Order book modeling and financial stability," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 14(3), pages 469-489, September.
    4. Alessio Emanuele Biondo, 2018. "Order book microstructure and policies for financial stability," Studies in Economics and Finance, Emerald Group Publishing Limited, vol. 35(1), pages 196-218, March.
    5. Biondo, Alessio Emanuele, 2018. "Learning to forecast, risk aversion, and microstructural aspects of financial stability," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 12, pages 1-21.
    6. Biondo, Alessio Emanuele, 2017. "Learning to forecast, risk aversion, and microstructural aspects of financial stability," Economics Discussion Papers 2017-104, Kiel Institute for the World Economy (IfW Kiel).
    7. Frank McGroarty & Ash Booth & Enrico Gerding & V. L. Raju Chinthalapati, 2019. "High frequency trading strategies, market fragility and price spikes: an agent based model perspective," Annals of Operations Research, Springer, vol. 282(1), pages 217-244, November.
    8. Alessio Emanuele Biondo, 2020. "Information versus imitation in a real-time agent-based model of financial markets," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 15(3), pages 613-631, July.
    9. Marcello Rambaldi & Emmanuel Bacry & Jean-Franc{c}ois Muzy, 2018. "Disentangling and quantifying market participant volatility contributions," Papers 1807.07036, arXiv.org.
    10. Ulrich Horst & Michael Paulsen, 2017. "A Law of Large Numbers for Limit Order Books," Mathematics of Operations Research, INFORMS, vol. 42(4), pages 1280-1312, November.
    11. Alexis Stenfors & Masayuki Susai, 2021. "Stealth Trading in FX Markets," Working Papers in Economics & Finance 2021-02, University of Portsmouth, Portsmouth Business School, Economics and Finance Subject Group.
    12. Weibing Huang & Charles-Albert Lehalle & Mathieu Rosenbaum, 2015. "Simulating and Analyzing Order Book Data: The Queue-Reactive Model," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(509), pages 107-122, March.
    13. Michael Frömmel & Frederick Van Gysegem, 2012. "Spread Components in the Hungarian Forint-Euro Market," Emerging Markets Finance and Trade, Taylor & Francis Journals, vol. 48(3), pages 52-69, May.
    14. Danny Lo, 2015. "Essays in Market Microstructure and Investor Trading," PhD Thesis, Finance Discipline Group, UTS Business School, University of Technology, Sydney, number 4-2015.
    15. Weibing Huang & Mathieu Rosenbaum & Pamela Saliba, 2019. "From Glosten-Milgrom to the whole limit order book and applications to financial regulation," Papers 1902.10743, arXiv.org.
    16. Vayanos, Dimitri & Wang, Jiang, 2013. "Market Liquidity—Theory and Empirical Evidence ," Handbook of the Economics of Finance, in: G.M. Constantinides & M. Harris & R. M. Stulz (ed.), Handbook of the Economics of Finance, volume 2, chapter 0, pages 1289-1361, Elsevier.
    17. Murphy Jun Jie Lee, 2013. "The Microstructure of Trading Processes on the Singapore Exchange," PhD Thesis, Finance Discipline Group, UTS Business School, University of Technology, Sydney, number 2-2013.
    18. Roberto Pascual & David Veredas, 2010. "Does the Open Limit Order Book Matter in Explaining Informational Volatility?," Journal of Financial Econometrics, Oxford University Press, vol. 8(1), pages 57-87, Winter.
    19. Julius Bonart & Martin D. Gould, 2017. "Latency and liquidity provision in a limit order book," Quantitative Finance, Taylor & Francis Journals, vol. 17(10), pages 1601-1616, October.
    20. Alexander Lykov & Stepan Muzychka & Kirill Vaninsky, 2016. "Investor'S Sentiment In Multi-Agent Model Of The Continuous Double Auction," International Journal of Theoretical and Applied Finance (IJTAF), World Scientific Publishing Co. Pte. Ltd., vol. 19(06), pages 1-29, September.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2403.19781. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.