IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2110.14771.html
   My bibliography  Save this paper

ABIDES-Gym: Gym Environments for Multi-Agent Discrete Event Simulation and Application to Financial Markets

Author

Listed:
  • Selim Amrouni
  • Aymeric Moulin
  • Jared Vann
  • Svitlana Vyetrenko
  • Tucker Balch
  • Manuela Veloso

Abstract

Model-free Reinforcement Learning (RL) requires the ability to sample trajectories by taking actions in the original problem environment or a simulated version of it. Breakthroughs in the field of RL have been largely facilitated by the development of dedicated open source simulators with easy to use frameworks such as OpenAI Gym and its Atari environments. In this paper we propose to use the OpenAI Gym framework on discrete event time based Discrete Event Multi-Agent Simulation (DEMAS). We introduce a general technique to wrap a DEMAS simulator into the Gym framework. We expose the technique in detail and implement it using the simulator ABIDES as a base. We apply this work by specifically using the markets extension of ABIDES, ABIDES-Markets, and develop two benchmark financial markets OpenAI Gym environments for training daily investor and execution agents. As a result, these two environments describe classic financial problems with a complex interactive market behavior response to the experimental agent's action.

Suggested Citation

  • Selim Amrouni & Aymeric Moulin & Jared Vann & Svitlana Vyetrenko & Tucker Balch & Manuela Veloso, 2021. "ABIDES-Gym: Gym Environments for Multi-Agent Discrete Event Simulation and Application to Financial Markets," Papers 2110.14771, arXiv.org.
    • Handle: RePEc:arx:papers:2110.14771
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2110.14771
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Nelson Minar & Rogert Burkhart & Chris Langton & Manor Askenazi, 1996. "The Swarm Simulation System: A Toolkit for Building Multi-Agent Simulations," Working Papers 96-06-042, Santa Fe Institute.
    2. R.J. Aumann & S. Hart (ed.), 2002. "Handbook of Game Theory with Economic Applications," Handbook of Game Theory with Economic Applications, Elsevier, edition 1, volume 3, number 3.
    3. Tucker Hybinette Balch & Mahmoud Mahfouz & Joshua Lockhart & Maria Hybinette & David Byrd, 2019. "How to Evaluate Trading Strategies: Single Agent Market Replay or Multiple Agent Interactive Simulation?," Papers 1906.12010, arXiv.org.
    4. Svitlana Vyetrenko & David Byrd & Nick Petosa & Mahmoud Mahfouz & Danial Dervovic & Manuela Veloso & Tucker Hybinette Balch, 2019. "Get Real: Realism Metrics for Robust Limit Order Book Market Simulations," Papers 1912.04941, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Selim Amrouni & Aymeric Moulin & Tucker Balch, 2022. "CTMSTOU driven markets: simulated environment for regime-awareness in trading policies," Papers 2202.00941, arXiv.org, revised Feb 2022.
    2. Saizhuo Wang & Hang Yuan & Leon Zhou & Lionel M. Ni & Heung-Yeung Shum & Jian Guo, 2023. "Alpha-GPT: Human-AI Interactive Alpha Mining for Quantitative Investment," Papers 2308.00016, arXiv.org.
    3. Penghang Liu & Kshama Dwarakanath & Svitlana S Vyetrenko & Tucker Balch, 2022. "Limited or Biased: Modeling Sub-Rational Human Investors in Financial Markets," Papers 2210.08569, arXiv.org, revised Mar 2024.
    4. Nicolas Cofre & Magdalena Mosionek-Schweda, 2023. "A simulated electronic market with speculative behaviour and bubble formation," Papers 2311.12247, arXiv.org.
    5. Joseph Jerome & Leandro Sanchez-Betancourt & Rahul Savani & Martin Herdegen, 2022. "Model-based gym environments for limit order book trading," Papers 2209.07823, arXiv.org.
    6. Song Wei & Andrea Coletta & Svitlana Vyetrenko & Tucker Balch, 2023. "INTAGS: Interactive Agent-Guided Simulation," Papers 2309.01784, arXiv.org, revised Nov 2023.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michael Karpe, 2020. "An overall view of key problems in algorithmic trading and recent progress," Papers 2006.05515, arXiv.org.
    2. Bruno Gašperov & Stjepan Begušić & Petra Posedel Šimović & Zvonko Kostanjčar, 2021. "Reinforcement Learning Approaches to Optimal Market Making," Mathematics, MDPI, vol. 9(21), pages 1-22, October.
    3. Marco Faravelli & Randall Walsh, 2011. "Smooth Politicians And Paternalistic Voters: A Theory Of Large Elections," Levine's Working Paper Archive 786969000000000250, David K. Levine.
    4. Christoph Schlueter-Langdon, 2000. "Information Technology And The Vertical Organization Of Industry," Computing in Economics and Finance 2000 174, Society for Computational Economics.
    5. Ding, Zhanwen & Shi, Guiping, 2009. "Cooperation in a dynamical adjustment of duopoly game with incomplete information," Chaos, Solitons & Fractals, Elsevier, vol. 42(2), pages 989-993.
    6. Luís de Sousa & Alberto Rodrigues da Silva, 2015. "Showcasing a Domain Specific Language for Spatial Simulation Scenarios with case studies," ERSA conference papers ersa15p1044, European Regional Science Association.
    7. Jhinyoung Shin & Rajdeep Singh, 2010. "Corporate Disclosures: Strategic Donation of Information," International Review of Finance, International Review of Finance Ltd., vol. 10(3), pages 313-337, September.
    8. Mario Guajardo & Kurt Jörnsten & Mikael Rönnqvist, 2016. "Constructive and blocking power in collaborative transportation," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 38(1), pages 25-50, January.
    9. Sylvie Thoron, 2016. "Morality Beyond Social Preferences: Smithian Sympathy, Social Neuroscience and the Nature of Social Consciousness [La moralité au delà des préférences sociales. La sympathie Smithienne, les neurosc," Post-Print hal-01645043, HAL.
    10. Antonio Briola & Jeremy Turiel & Riccardo Marcaccioli & Alvaro Cauderan & Tomaso Aste, 2021. "Deep Reinforcement Learning for Active High Frequency Trading," Papers 2101.07107, arXiv.org, revised Aug 2023.
    11. Nan Xia & S. Rajagopalan, 2009. "Standard vs. Custom Products: Variety, Lead Time, and Price Competition," Marketing Science, INFORMS, vol. 28(5), pages 887-900, 09-10.
    12. Gerard Llobet & Javier Suarez, 2010. "Entrepreneurial Innovation, Patent Protection and Industry Dynamics," Working Papers wp2010_1001, CEMFI.
    13. Leo Ardon & Nelson Vadori & Thomas Spooner & Mengda Xu & Jared Vann & Sumitra Ganesh, 2021. "Towards a fully RL-based Market Simulator," Papers 2110.06829, arXiv.org, revised Nov 2021.
    14. Platz, Trine Tornøe & Østerdal, Lars Peter, 2017. "The curse of the first-in–first-out queue discipline," Games and Economic Behavior, Elsevier, vol. 104(C), pages 165-176.
    15. Jihui Chen & Qiang Fu, 2017. "Do exclusivity arrangements harm consumers?," Journal of Regulatory Economics, Springer, vol. 51(3), pages 311-339, June.
    16. Dinah Rosenberg & Eilon Solan & Nicolas Vieille, 2009. "Protocols with No Acknowledgment," Operations Research, INFORMS, vol. 57(4), pages 905-915, August.
    17. Yusuke Kamishiro & Roberto Serrano & Myrna Wooders, 2021. "Monopolists of scarce information and small group effectiveness in large quasilinear economies," International Journal of Game Theory, Springer;Game Theory Society, vol. 50(4), pages 801-827, December.
    18. Tristan Tomala, 2011. "Fault Reporting in Partially Known Networks and Folk Theorems," Operations Research, INFORMS, vol. 59(3), pages 754-763, June.
    19. Sen, Debapriya & Tauman, Yair, 2007. "General licensing schemes for a cost-reducing innovation," Games and Economic Behavior, Elsevier, vol. 59(1), pages 163-186, April.
    20. Sent, Esther-Mirjam, 2004. "The legacy of Herbert Simon in game theory," Journal of Economic Behavior & Organization, Elsevier, vol. 53(3), pages 303-317, March.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2110.14771. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.