IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2502.09172.html
   My bibliography  Save this paper

LOB-Bench: Benchmarking Generative AI for Finance -- an Application to Limit Order Book Data

Author

Listed:
  • Peer Nagy
  • Sascha Frey
  • Kang Li
  • Bidipta Sarkar
  • Svitlana Vyetrenko
  • Stefan Zohren
  • Ani Calinescu
  • Jakob Foerster

Abstract

While financial data presents one of the most challenging and interesting sequence modelling tasks due to high noise, heavy tails, and strategic interactions, progress in this area has been hindered by the lack of consensus on quantitative evaluation paradigms. To address this, we present LOB-Bench, a benchmark, implemented in python, designed to evaluate the quality and realism of generative message-by-order data for limit order books (LOB) in the LOBSTER format. Our framework measures distributional differences in conditional and unconditional statistics between generated and real LOB data, supporting flexible multivariate statistical evaluation. The benchmark also includes features commonly used LOB statistics such as spread, order book volumes, order imbalance, and message inter-arrival times, along with scores from a trained discriminator network. Lastly, LOB-Bench contains "market impact metrics", i.e. the cross-correlations and price response functions for specific events in the data. We benchmark generative autoregressive state-space models, a (C)GAN, as well as a parametric LOB model and find that the autoregressive GenAI approach beats traditional model classes.

Suggested Citation

  • Peer Nagy & Sascha Frey & Kang Li & Bidipta Sarkar & Svitlana Vyetrenko & Stefan Zohren & Ani Calinescu & Jakob Foerster, 2025. "LOB-Bench: Benchmarking Generative AI for Finance -- an Application to Limit Order Book Data," Papers 2502.09172, arXiv.org, revised Jun 2025.
    • Handle: RePEc:arx:papers:2502.09172
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2502.09172
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Rama Cont & Arseniy Kukanov & Sasha Stoikov, 2014. "The Price Impact of Order Book Events," Journal of Financial Econometrics, Oxford University Press, vol. 12(1), pages 47-88.
    2. Rama Cont & Sasha Stoikov & Rishi Talreja, 2010. "A Stochastic Model for Order Book Dynamics," Operations Research, INFORMS, vol. 58(3), pages 549-563, June.
    3. Carl Chiarella & Giulia Iori, 2002. "A simulation analysis of the microstructure of double auction markets," Quantitative Finance, Taylor & Francis Journals, vol. 2(5), pages 346-353.
    4. Bence Toth & Zoltan Eisler & Jean-Philippe Bouchaud, 2016. "The square-root impact law also holds for option markets," Papers 1602.03043, arXiv.org.
    5. Peer Nagy & Sascha Frey & Silvia Sapora & Kang Li & Anisoara Calinescu & Stefan Zohren & Jakob Foerster, 2023. "Generative AI for End-to-End Limit Order Book Modelling: A Token-Level Autoregressive Generative Model of Message Flow Using a Deep State Space Network," Papers 2309.00638, arXiv.org.
    6. Zoltán Eisler & Jean-Philippe Bouchaud & Julien Kockelkoren, 2012. "The price impact of order book events: market orders, limit orders and cancellations," Quantitative Finance, Taylor & Francis Journals, vol. 12(9), pages 1395-1419, September.
    7. R. Cont, 2001. "Empirical properties of asset returns: stylized facts and statistical issues," Quantitative Finance, Taylor & Francis Journals, vol. 1(2), pages 223-236.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mateusz Wilinski & Juho Kanniainen, 2025. "Prospects of Imitating Trading Agents in the Stock Market," Papers 2509.00982, arXiv.org.
    2. Jaskaran Singh Walia & Aarush Sinha & Srinitish Srinivasan & Srihari Unnikrishnan, 2025. "Predicting Liquidity-Aware Bond Yields using Causal GANs and Deep Reinforcement Learning with LLM Evaluation," Papers 2502.17011, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Martin D. Gould & Mason A. Porter & Stacy Williams & Mark McDonald & Daniel J. Fenn & Sam D. Howison, 2010. "Limit Order Books," Papers 1012.0349, arXiv.org, revised Apr 2013.
    2. Justin Sirignano & Rama Cont, 2018. "Universal features of price formation in financial markets: perspectives from Deep Learning," Papers 1803.06917, arXiv.org.
    3. Justin Sirignano & Rama Cont, 2018. "Universal features of price formation in financial markets: perspectives from Deep Learning," Working Papers hal-01754054, HAL.
    4. Fabrizio Lillo, 2021. "Order flow and price formation," Papers 2105.00521, arXiv.org.
    5. Leonardo Berti & Bardh Prenkaj & Paola Velardi, 2025. "TRADES: Generating Realistic Market Simulations with Diffusion Models," Papers 2502.07071, arXiv.org, revised Nov 2025.
    6. Martin D. Gould & Mason A. Porter & Stacy Williams & Mark McDonald & Daniel J. Fenn & Sam D. Howison, 2013. "Limit order books," Quantitative Finance, Taylor & Francis Journals, vol. 13(11), pages 1709-1742, November.
    7. Zijian Shi & John Cartlidge, 2024. "Neural stochastic agent‐based limit order book simulation with neural point process and diffusion probabilistic model," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 31(2), June.
    8. Przemys{l}aw Rola, 2025. "Boltzmann Price: Toward Understanding the Fair Price in High-Frequency Markets," Papers 2507.09734, arXiv.org.
    9. Tucker Hybinette Balch & Mahmoud Mahfouz & Joshua Lockhart & Maria Hybinette & David Byrd, 2019. "How to Evaluate Trading Strategies: Single Agent Market Replay or Multiple Agent Interactive Simulation?," Papers 1906.12010, arXiv.org.
    10. Alessio Emanuele Biondo, 2019. "Order book modeling and financial stability," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 14(3), pages 469-489, September.
    11. Federico Gonzalez & Mark Schervish, 2017. "Instantaneous order impact and high-frequency strategy optimization in limit order books," Papers 1707.01167, arXiv.org, revised Oct 2017.
    12. Zijian Shi & John Cartlidge, 2023. "Neural Stochastic Agent-Based Limit Order Book Simulation: A Hybrid Methodology," Papers 2303.00080, arXiv.org.
    13. Biondo, Alessio Emanuele, 2018. "Learning to forecast, risk aversion, and microstructural aspects of financial stability," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 12, pages 1-21.
    14. Alessio Emanuele Biondo, 2018. "Order book microstructure and policies for financial stability," Studies in Economics and Finance, Emerald Group Publishing Limited, vol. 35(1), pages 196-218, March.
    15. Mircea BAHNA & Cosmin-Octavian CEPOI & Bogdan Andrei DUMITRESCU & Virgil DAMIAN, 2018. "Estimating the Price Impact of Market Orders on the Bucharest Stock Exchange," Journal for Economic Forecasting, Institute for Economic Forecasting, vol. 0(4), pages 120-133, December.
    16. Petter N. Kolm & Jeremy Turiel & Nicholas Westray, 2023. "Deep order flow imbalance: Extracting alpha at multiple horizons from the limit order book," Mathematical Finance, Wiley Blackwell, vol. 33(4), pages 1044-1081, October.
    17. Biondo, Alessio Emanuele, 2017. "Learning to forecast, risk aversion, and microstructural aspects of financial stability," Economics Discussion Papers 2017-104, Kiel Institute for the World Economy (IfW Kiel).
    18. Frank McGroarty & Ash Booth & Enrico Gerding & V. L. Raju Chinthalapati, 2019. "High frequency trading strategies, market fragility and price spikes: an agent based model perspective," Annals of Operations Research, Springer, vol. 282(1), pages 217-244, November.
    19. Michael Giegrich & Roel Oomen & Christoph Reisinger, 2024. "Limit Order Book Simulation and Trade Evaluation with $K$-Nearest-Neighbor Resampling," Papers 2409.06514, arXiv.org.
    20. Stanis{l}aw Dro.zd.z & Jaros{l}aw Kwapie'n & Marcin Wk{a}torek, 2023. "What is mature and what is still emerging in the cryptocurrency market?," Papers 2305.05751, arXiv.org.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2502.09172. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.