IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2110.06829.html
   My bibliography  Save this paper

Towards a fully RL-based Market Simulator

Author

Listed:
  • Leo Ardon
  • Nelson Vadori
  • Thomas Spooner
  • Mengda Xu
  • Jared Vann
  • Sumitra Ganesh

Abstract

We present a new financial framework where two families of RL-based agents representing the Liquidity Providers and Liquidity Takers learn simultaneously to satisfy their objective. Thanks to a parametrized reward formulation and the use of Deep RL, each group learns a shared policy able to generalize and interpolate over a wide range of behaviors. This is a step towards a fully RL-based market simulator replicating complex market conditions particularly suited to study the dynamics of the financial market under various scenarios.

Suggested Citation

  • Leo Ardon & Nelson Vadori & Thomas Spooner & Mengda Xu & Jared Vann & Sumitra Ganesh, 2021. "Towards a fully RL-based Market Simulator," Papers 2110.06829, arXiv.org, revised Nov 2021.
    • Handle: RePEc:arx:papers:2110.06829
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2110.06829
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Olivier Gu'eant & Charles-Albert Lehalle & Joaquin Fernandez Tapia, 2011. "Dealing with the Inventory Risk. A solution to the market making problem," Papers 1105.3115, arXiv.org, revised Aug 2012.
    2. Thomas Spooner & Rahul Savani, 2020. "Robust Market Making via Adversarial Reinforcement Learning," Papers 2003.01820, arXiv.org, revised Jul 2020.
    3. Svitlana Vyetrenko & David Byrd & Nick Petosa & Mahmoud Mahfouz & Danial Dervovic & Manuela Veloso & Tucker Hybinette Balch, 2019. "Get Real: Realism Metrics for Robust Limit Order Book Market Simulations," Papers 1912.04941, arXiv.org.
    4. Amihud, Yakov & Mendelson, Haim, 1980. "Dealership market : Market-making with inventory," Journal of Financial Economics, Elsevier, vol. 8(1), pages 31-53, March.
    5. Garman, Mark B., 1976. "Market microstructure," Journal of Financial Economics, Elsevier, vol. 3(3), pages 257-275, June.
    6. Marco Avellaneda & Sasha Stoikov, 2008. "High-frequency trading in a limit order book," Quantitative Finance, Taylor & Francis Journals, vol. 8(3), pages 217-224.
    7. Nicholas T. Chan and Christian Shelton, 2001. "An Adaptive Electronic Market-Maker," Computing in Economics and Finance 2001 146, Society for Computational Economics.
    8. Sumitra Ganesh & Nelson Vadori & Mengda Xu & Hua Zheng & Prashant Reddy & Manuela Veloso, 2019. "Reinforcement Learning for Market Making in a Multi-agent Dealer Market," Papers 1911.05892, arXiv.org.
    9. Rama Cont & Marvin S. Mueller, 2019. "A stochastic partial differential equation model for limit order book dynamics," Papers 1904.03058, arXiv.org, revised May 2021.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nelson Vadori & Leo Ardon & Sumitra Ganesh & Thomas Spooner & Selim Amrouni & Jared Vann & Mengda Xu & Zeyu Zheng & Tucker Balch & Manuela Veloso, 2022. "Towards Multi-Agent Reinforcement Learning driven Over-The-Counter Market Simulations," Papers 2210.07184, arXiv.org, revised Aug 2023.
    2. Xiao-Yang Liu & Ziyi Xia & Jingyang Rui & Jiechao Gao & Hongyang Yang & Ming Zhu & Christina Dan Wang & Zhaoran Wang & Jian Guo, 2022. "FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning," Papers 2211.03107, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nelson Vadori & Leo Ardon & Sumitra Ganesh & Thomas Spooner & Selim Amrouni & Jared Vann & Mengda Xu & Zeyu Zheng & Tucker Balch & Manuela Veloso, 2022. "Towards Multi-Agent Reinforcement Learning driven Over-The-Counter Market Simulations," Papers 2210.07184, arXiv.org, revised Aug 2023.
    2. Joseph Jerome & Leandro Sanchez-Betancourt & Rahul Savani & Martin Herdegen, 2022. "Model-based gym environments for limit order book trading," Papers 2209.07823, arXiv.org.
    3. Bruno Gašperov & Stjepan Begušić & Petra Posedel Šimović & Zvonko Kostanjčar, 2021. "Reinforcement Learning Approaches to Optimal Market Making," Mathematics, MDPI, vol. 9(21), pages 1-22, October.
    4. Sumitra Ganesh & Nelson Vadori & Mengda Xu & Hua Zheng & Prashant Reddy & Manuela Veloso, 2019. "Reinforcement Learning for Market Making in a Multi-agent Dealer Market," Papers 1911.05892, arXiv.org.
    5. Marina Di Giacinto & Claudio Tebaldi & Tai-Ho Wang, 2021. "Optimal order execution under price impact: A hybrid model," Papers 2112.02228, arXiv.org, revised Aug 2022.
    6. Christoph Kuhn & Johannes Muhle-Karbe, 2013. "Optimal Liquidity Provision," Papers 1309.5235, arXiv.org, revised Feb 2015.
    7. Hui Niu & Siyuan Li & Jiahao Zheng & Zhouchi Lin & Jian Li & Jian Guo & Bo An, 2023. "IMM: An Imitative Reinforcement Learning Approach with Predictive Representation Learning for Automatic Market Making," Papers 2308.08918, arXiv.org.
    8. M. Alessandra Crisafi & Andrea Macrina, 2015. "Dark-Pool Perspective of Optimal Market Making," Papers 1502.02863, arXiv.org.
    9. Kühn, Christoph & Muhle-Karbe, Johannes, 2015. "Optimal liquidity provision," Stochastic Processes and their Applications, Elsevier, vol. 125(7), pages 2493-2515.
    10. Baron Law & Frederi Viens, 2019. "Market Making under a Weakly Consistent Limit Order Book Model," Papers 1903.07222, arXiv.org, revised Jan 2020.
    11. Hong Guo & Jianwu Lin & Fanlin Huang, 2023. "Market Making with Deep Reinforcement Learning from Limit Order Books," Papers 2305.15821, arXiv.org.
    12. Ben Hambly & Renyuan Xu & Huining Yang, 2021. "Recent Advances in Reinforcement Learning in Finance," Papers 2112.04553, arXiv.org, revised Feb 2023.
    13. Philippe Bergault & Louis Bertucci & David Bouba & Olivier Gu'eant, 2022. "Automated Market Makers: Mean-Variance Analysis of LPs Payoffs and Design of Pricing Functions," Papers 2212.00336, arXiv.org, revised Nov 2023.
    14. Shuo Sun & Rundong Wang & Bo An, 2021. "Reinforcement Learning for Quantitative Trading," Papers 2109.13851, arXiv.org.
    15. Ali Raheman & Anton Kolonin & Alexey Glushchenko & Arseniy Fokin & Ikram Ansari, 2022. "Adaptive Multi-Strategy Market-Making Agent For Volatile Markets," Papers 2204.13265, arXiv.org.
    16. Bruno Gav{s}perov & Zvonko Kostanjv{c}ar, 2022. "Deep Reinforcement Learning for Market Making Under a Hawkes Process-Based Limit Order Book Model," Papers 2207.09951, arXiv.org.
    17. Pankaj Kumar, 2021. "Deep Hawkes Process for High-Frequency Market Making," Papers 2109.15110, arXiv.org.
    18. Joseph Jerome & Gregory Palmer & Rahul Savani, 2022. "Market Making with Scaled Beta Policies," Papers 2207.03352, arXiv.org, revised Sep 2022.
    19. Ben Hambly & Renyuan Xu & Huining Yang, 2023. "Recent advances in reinforcement learning in finance," Mathematical Finance, Wiley Blackwell, vol. 33(3), pages 437-503, July.
    20. Zhou Fang & Haiqing Xu, 2023. "Market Making of Options via Reinforcement Learning," Papers 2307.01814, arXiv.org.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2110.06829. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.