IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1906.12010.html
   My bibliography  Save this paper

How to Evaluate Trading Strategies: Single Agent Market Replay or Multiple Agent Interactive Simulation?

Author

Listed:
  • Tucker Hybinette Balch
  • Mahmoud Mahfouz
  • Joshua Lockhart
  • Maria Hybinette
  • David Byrd

Abstract

We show how a multi-agent simulator can support two important but distinct methods for assessing a trading strategy: Market Replay and Interactive Agent-Based Simulation (IABS). Our solution is important because each method offers strengths and weaknesses that expose or conceal flaws in the subject strategy. A key weakness of Market Replay is that the simulated market does not substantially adapt to or respond to the presence of the experimental strategy. IABS methods provide an artificial market for the experimental strategy using a population of background trading agents. Because the background agents attend to market conditions and current price as part of their strategy, the overall market is responsive to the presence of the experimental strategy. Even so, IABS methods have their own weaknesses, primarily that it is unclear if the market environment they provide is realistic. We describe our approach in detail, and illustrate its use in an example application: The evaluation of market impact for various size orders.

Suggested Citation

  • Tucker Hybinette Balch & Mahmoud Mahfouz & Joshua Lockhart & Maria Hybinette & David Byrd, 2019. "How to Evaluate Trading Strategies: Single Agent Market Replay or Multiple Agent Interactive Simulation?," Papers 1906.12010, arXiv.org.
    • Handle: RePEc:arx:papers:1906.12010
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1906.12010
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Martin D. Gould & Mason A. Porter & Stacy Williams & Mark McDonald & Daniel J. Fenn & Sam D. Howison, 2010. "Limit Order Books," Papers 1012.0349, arXiv.org, revised Apr 2013.
    2. Rama Cont & Arseniy Kukanov & Sasha Stoikov, 2014. "The Price Impact of Order Book Events," Journal of Financial Econometrics, Oxford University Press, vol. 12(1), pages 47-88.
    3. Rama Cont & Sasha Stoikov & Rishi Talreja, 2010. "A Stochastic Model for Order Book Dynamics," Operations Research, INFORMS, vol. 58(3), pages 549-563, June.
    4. Gode, Dhananjay K & Sunder, Shyam, 1993. "Allocative Efficiency of Markets with Zero-Intelligence Traders: Market as a Partial Substitute for Individual Rationality," Journal of Political Economy, University of Chicago Press, vol. 101(1), pages 119-137, February.
    5. Martin D. Gould & Mason A. Porter & Stacy Williams & Mark McDonald & Daniel J. Fenn & Sam D. Howison, 2013. "Limit order books," Quantitative Finance, Taylor & Francis Journals, vol. 13(11), pages 1709-1742, November.
    6. R. Cont, 2001. "Empirical properties of asset returns: stylized facts and statistical issues," Quantitative Finance, Taylor & Francis Journals, vol. 1(2), pages 223-236.
    7. John Duffy & M. Ünver, 2006. "Asset price bubbles and crashes with near-zero-intelligence traders," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 27(3), pages 537-563, April.
    8. Kyle, Albert S, 1985. "Continuous Auctions and Insider Trading," Econometrica, Econometric Society, vol. 53(6), pages 1315-1335, November.
    9. LeBaron, Blake, 2006. "Agent-based Computational Finance," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 24, pages 1187-1233, Elsevier.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hamza Bodor & Laurent Carlier, 2024. "A Novel Approach to Queue-Reactive Models: The Importance of Order Sizes," Papers 2405.18594, arXiv.org.
    2. Andrea Coletta & Joseph Jerome & Rahul Savani & Svitlana Vyetrenko, 2023. "Conditional Generators for Limit Order Book Environments: Explainability, Challenges, and Robustness," Papers 2306.12806, arXiv.org.
    3. Andrea Coletta & Aymeric Moulin & Svitlana Vyetrenko & Tucker Balch, 2022. "Learning to simulate realistic limit order book markets from data as a World Agent," Papers 2210.09897, arXiv.org.
    4. Yadh Hafsi & Edoardo Vittori, 2024. "Optimal Execution with Reinforcement Learning," Papers 2411.06389, arXiv.org.
    5. Yu-Hao Huang & Chang Xu & Yang Liu & Weiqing Liu & Wu-Jun Li & Jiang Bian, 2024. "Controllable Financial Market Generation with Diffusion Guided Meta Agent," Papers 2408.12991, arXiv.org, revised Sep 2024.
    6. Antonio Briola & Jeremy Turiel & Riccardo Marcaccioli & Alvaro Cauderan & Tomaso Aste, 2021. "Deep Reinforcement Learning for Active High Frequency Trading," Papers 2101.07107, arXiv.org, revised Aug 2023.
    7. Bruno Gašperov & Stjepan Begušić & Petra Posedel Šimović & Zvonko Kostanjčar, 2021. "Reinforcement Learning Approaches to Optimal Market Making," Mathematics, MDPI, vol. 9(21), pages 1-22, October.
    8. Michael Karpe, 2020. "An overall view of key problems in algorithmic trading and recent progress," Papers 2006.05515, arXiv.org.
    9. Selim Amrouni & Aymeric Moulin & Jared Vann & Svitlana Vyetrenko & Tucker Balch & Manuela Veloso, 2021. "ABIDES-Gym: Gym Environments for Multi-Agent Discrete Event Simulation and Application to Financial Markets," Papers 2110.14771, arXiv.org.
    10. Andrea Coletta & Matteo Prata & Michele Conti & Emanuele Mercanti & Novella Bartolini & Aymeric Moulin & Svitlana Vyetrenko & Tucker Balch, 2021. "Towards Realistic Market Simulations: a Generative Adversarial Networks Approach," Papers 2110.13287, arXiv.org.
    11. Joseph Jerome & Gregory Palmer & Rahul Savani, 2022. "Market Making with Scaled Beta Policies," Papers 2207.03352, arXiv.org, revised Sep 2022.
    12. Penghang Liu & Kshama Dwarakanath & Svitlana S Vyetrenko & Tucker Balch, 2022. "Limited or Biased: Modeling Sub-Rational Human Investors in Financial Markets," Papers 2210.08569, arXiv.org, revised Mar 2024.
    13. Nicolas Cofre & Magdalena Mosionek-Schweda, 2023. "A simulated electronic market with speculative behaviour and bubble formation," Papers 2311.12247, arXiv.org.
    14. Marcello Monga, 2024. "Automated Market Making and Decentralized Finance," Papers 2407.16885, arXiv.org.
    15. Joseph Jerome & Leandro Sanchez-Betancourt & Rahul Savani & Martin Herdegen, 2022. "Model-based gym environments for limit order book trading," Papers 2209.07823, arXiv.org.
    16. David Byrd, 2019. "Explaining Agent-Based Financial Market Simulation," Papers 1909.11650, arXiv.org.
    17. Song Wei & Andrea Coletta & Svitlana Vyetrenko & Tucker Balch, 2023. "INTAGS: Interactive Agent-Guided Simulation," Papers 2309.01784, arXiv.org, revised Nov 2023.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alberto Ciacci & Takumi Sueshige & Hideki Takayasu & Kim Christensen & Misako Takayasu, 2020. "The microscopic relationships between triangular arbitrage and cross-currency correlations in a simple agent based model of foreign exchange markets," PLOS ONE, Public Library of Science, vol. 15(6), pages 1-19, June.
    2. Hai-Chuan Xu & Wei Chen & Xiong Xiong & Wei Zhang & Wei-Xing Zhou & H Eugene Stanley, 2016. "Limit-order book resiliency after effective market orders: Spread, depth and intensity," Papers 1602.00731, arXiv.org, revised Feb 2017.
    3. Martin D. Gould & Mason A. Porter & Stacy Williams & Mark McDonald & Daniel J. Fenn & Sam D. Howison, 2010. "Limit Order Books," Papers 1012.0349, arXiv.org, revised Apr 2013.
    4. Ningyuan Chen & Steven Kou & Chun Wang, 2018. "A Partitioning Algorithm for Markov Decision Processes with Applications to Market Microstructure," Management Science, INFORMS, vol. 64(2), pages 784-803, February.
    5. Weibing Huang & Mathieu Rosenbaum & Pamela Saliba, 2019. "From Glosten-Milgrom to the whole limit order book and applications to financial regulation," Papers 1902.10743, arXiv.org.
    6. Alessio Emanuele Biondo, 2019. "Order book modeling and financial stability," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 14(3), pages 469-489, September.
    7. Ivan Jericevich & Patrick Chang & Tim Gebbie, 2021. "Simulation and estimation of an agent-based market-model with a matching engine," Papers 2108.07806, arXiv.org, revised Aug 2021.
    8. James Paulin & Anisoara Calinescu & Michael Wooldridge, 2018. "Understanding Flash Crash Contagion and Systemic Risk: A Micro-Macro Agent-Based Approach," Papers 1805.08454, arXiv.org.
    9. Federico Gonzalez & Mark Schervish, 2017. "Instantaneous order impact and high-frequency strategy optimization in limit order books," Papers 1707.01167, arXiv.org, revised Oct 2017.
    10. Paulin, James & Calinescu, Anisoara & Wooldridge, Michael, 2019. "Understanding flash crash contagion and systemic risk: A micro–macro agent-based approach," Journal of Economic Dynamics and Control, Elsevier, vol. 100(C), pages 200-229.
    11. Biondo, Alessio Emanuele, 2018. "Learning to forecast, risk aversion, and microstructural aspects of financial stability," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 12, pages 1-21.
    12. Antoine Jacquier & Hao Liu, 2017. "Optimal liquidation in a Level-I limit order book for large tick stocks," Papers 1701.01327, arXiv.org, revised Nov 2017.
    13. Ben Hambly & Jasdeep Kalsi & James Newbury, 2018. "Limit order books, diffusion approximations and reflected SPDEs: from microscopic to macroscopic models," Papers 1808.07107, arXiv.org, revised Jun 2019.
    14. Alessio Emanuele Biondo, 2018. "Order book microstructure and policies for financial stability," Studies in Economics and Finance, Emerald Group Publishing Limited, vol. 35(1), pages 196-218, March.
    15. Biondo, Alessio Emanuele, 2017. "Learning to forecast, risk aversion, and microstructural aspects of financial stability," Economics Discussion Papers 2017-104, Kiel Institute for the World Economy (IfW Kiel).
    16. Efstathios Panayi & Gareth W. Peters, 2015. "Stochastic simulation framework for the limit order book using liquidity-motivated agents," International Journal of Financial Engineering (IJFE), World Scientific Publishing Co. Pte. Ltd., vol. 2(02), pages 1-52.
    17. Martin D. Gould & Mason A. Porter & Stacy Williams & Mark McDonald & Daniel J. Fenn & Sam D. Howison, 2013. "Limit order books," Quantitative Finance, Taylor & Francis Journals, vol. 13(11), pages 1709-1742, November.
    18. Efstathios Panayi & Gareth Peters, 2015. "Stochastic simulation framework for the Limit Order Book using liquidity motivated agents," Papers 1501.02447, arXiv.org, revised Jan 2015.
    19. Radu T. Pruna & Maria Polukarov & Nicholas R. Jennings, 2016. "A new structural stochastic volatility model of asset pricing and its stylized facts," Papers 1604.08824, arXiv.org.
    20. Makarewicz, Tomasz, 2021. "Traders, forecasters and financial instability: A model of individual learning of anchor-and-adjustment heuristics," Journal of Economic Behavior & Organization, Elsevier, vol. 190(C), pages 626-673.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1906.12010. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.