IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2305.15821.html
   My bibliography  Save this paper

Market Making with Deep Reinforcement Learning from Limit Order Books

Author

Listed:
  • Hong Guo
  • Jianwu Lin
  • Fanlin Huang

Abstract

Market making (MM) is an important research topic in quantitative finance, the agent needs to continuously optimize ask and bid quotes to provide liquidity and make profits. The limit order book (LOB) contains information on all active limit orders, which is an essential basis for decision-making. The modeling of evolving, high-dimensional and low signal-to-noise ratio LOB data is a critical challenge. Traditional MM strategy relied on strong assumptions such as price process, order arrival process, etc. Previous reinforcement learning (RL) works handcrafted market features, which is insufficient to represent the market. This paper proposes a RL agent for market making with LOB data. We leverage a neural network with convolutional filters and attention mechanism (Attn-LOB) for feature extraction from LOB. We design a new continuous action space and a hybrid reward function for the MM task. Finally, we conduct comprehensive experiments on latency and interpretability, showing that our agent has good applicability.

Suggested Citation

  • Hong Guo & Jianwu Lin & Fanlin Huang, 2023. "Market Making with Deep Reinforcement Learning from Limit Order Books," Papers 2305.15821, arXiv.org.
    • Handle: RePEc:arx:papers:2305.15821
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2305.15821
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Dat Thanh Tran & Martin Magris & Juho Kanniainen & Moncef Gabbouj & Alexandros Iosifidis, 2017. "Tensor Representation in High-Frequency Financial Data for Price Change Prediction," Papers 1709.01268, arXiv.org, revised Nov 2017.
    2. Anirban Chakraborti & Ioane Muni Toke & Marco Patriarca & Frederic Abergel, 2011. "Econophysics review: I. Empirical facts," Quantitative Finance, Taylor & Francis Journals, vol. 11(7), pages 991-1012.
    3. Fabien Guilbaud & Huyên Pham, 2013. "Optimal high-frequency trading with limit and market orders," Quantitative Finance, Taylor & Francis Journals, vol. 13(1), pages 79-94, January.
    4. Ho, Thomas & Stoll, Hans R, 1980. "On Dealer Markets under Competition," Journal of Finance, American Finance Association, vol. 35(2), pages 259-267, May.
    5. Marco Avellaneda & Sasha Stoikov, 2008. "High-frequency trading in a limit order book," Quantitative Finance, Taylor & Francis Journals, vol. 8(3), pages 217-224.
    6. Glosten, Lawrence R, 1994. "Is the Electronic Open Limit Order Book Inevitable?," Journal of Finance, American Finance Association, vol. 49(4), pages 1127-1161, September.
    7. Adamantios Ntakaris & Giorgio Mirone & Juho Kanniainen & Moncef Gabbouj & Alexandros Iosifidis, 2019. "Feature Engineering for Mid-Price Prediction with Deep Learning," Papers 1904.05384, arXiv.org, revised Jun 2019.
    8. Harold Demsetz, 1968. "The Cost of Transacting," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 82(1), pages 33-53.
    9. A. Chakraborti & I. Muni-Toke & M. Patriarca & F. Abergel, 2011. "Econophysics Review : II. Agent-based models," Post-Print hal-03332946, HAL.
    10. Charles Cao & Oliver Hansch & Xiaoxin Wang, 2009. "The information content of an open limit‐order book," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 29(1), pages 16-41, January.
    11. Anirban Chakraborti & Ioane Muni Toke & Marco Patriarca & Frédéric Abergel, 2011. "Econophysics review: II. Agent-based models," Post-Print hal-00621059, HAL.
    12. Kyle, Albert S, 1985. "Continuous Auctions and Insider Trading," Econometrica, Econometric Society, vol. 53(6), pages 1315-1335, November.
    13. Antonio Briola & Jeremy Turiel & Riccardo Marcaccioli & Alvaro Cauderan & Tomaso Aste, 2021. "Deep Reinforcement Learning for Active High Frequency Trading," Papers 2101.07107, arXiv.org, revised Aug 2023.
    14. Thomas Spooner & John Fearnley & Rahul Savani & Andreas Koukorinis, 2018. "Market Making via Reinforcement Learning," Papers 1804.04216, arXiv.org.
    15. Jonathan Sadighian, 2019. "Deep Reinforcement Learning in Cryptocurrency Market Making," Papers 1911.08647, arXiv.org.
    16. Garman, Mark B., 1976. "Market microstructure," Journal of Financial Economics, Elsevier, vol. 3(3), pages 257-275, June.
    17. Justin Sirignano & Rama Cont, 2019. "Universal features of price formation in financial markets: perspectives from deep learning," Quantitative Finance, Taylor & Francis Journals, vol. 19(9), pages 1449-1459, September.
    18. Nicholas T. Chan and Christian Shelton, 2001. "An Adaptive Electronic Market-Maker," Computing in Economics and Finance 2001 146, Society for Computational Economics.
    19. Ho, Thomas & Stoll, Hans R., 1981. "Optimal dealer pricing under transactions and return uncertainty," Journal of Financial Economics, Elsevier, vol. 9(1), pages 47-73, March.
    20. Ioanid Rosu, 2009. "A Dynamic Model of the Limit Order Book," Post-Print hal-00515873, HAL.
    21. Anirban Chakraborti & Ioane Muni Toke & Marco Patriarca & Frederic Abergel, 2011. "Econophysics review: II. Agent-based models," Quantitative Finance, Taylor & Francis Journals, vol. 11(7), pages 1013-1041.
    22. Anirban Chakraborti & Ioane Muni Toke & Marco Patriarca & Frédéric Abergel, 2011. "Econophysics review: I. Empirical facts," Post-Print hal-00621058, HAL.
    23. Ioanid Rosu, 2009. "A Dynamic Model of the Limit Order Book," The Review of Financial Studies, Society for Financial Studies, vol. 22(11), pages 4601-4641, November.
    24. Zihao Zhang & Stefan Zohren & Stephen Roberts, 2019. "Extending Deep Learning Models for Limit Order Books to Quantile Regression," Papers 1906.04404, arXiv.org.
    25. Zihao Zhang & Stefan Zohren, 2021. "Multi-Horizon Forecasting for Limit Order Books: Novel Deep Learning Approaches and Hardware Acceleration using Intelligent Processing Units," Papers 2105.10430, arXiv.org, revised Aug 2021.
    26. Adamantios Ntakaris & Martin Magris & Juho Kanniainen & Moncef Gabbouj & Alexandros Iosifidis, 2018. "Benchmark dataset for mid‐price forecasting of limit order book data with machine learning methods," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 37(8), pages 852-866, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Vayanos, Dimitri & Wang, Jiang, 2013. "Market Liquidity—Theory and Empirical Evidence ," Handbook of the Economics of Finance, in: G.M. Constantinides & M. Harris & R. M. Stulz (ed.), Handbook of the Economics of Finance, volume 2, chapter 0, pages 1289-1361, Elsevier.
    2. Dimitri Vayanos & Jiang Wang, 2012. "Market Liquidity -- Theory and Empirical Evidence," NBER Working Papers 18251, National Bureau of Economic Research, Inc.
    3. Ivan Jericevich & Patrick Chang & Tim Gebbie, 2021. "Simulation and estimation of an agent-based market-model with a matching engine," Papers 2108.07806, arXiv.org, revised Aug 2021.
    4. Cenesizoglu, Tolga & Grass, Gunnar, 2018. "Bid- and ask-side liquidity in the NYSE limit order book," Journal of Financial Markets, Elsevier, vol. 38(C), pages 14-38.
    5. Alessio Emanuele Biondo, 2018. "Order book microstructure and policies for financial stability," Studies in Economics and Finance, Emerald Group Publishing Limited, vol. 35(1), pages 196-218, March.
    6. Biondo, Alessio Emanuele, 2017. "Learning to forecast, risk aversion, and microstructural aspects of financial stability," Economics Discussion Papers 2017-104, Kiel Institute for the World Economy (IfW Kiel).
    7. Bruno Gašperov & Stjepan Begušić & Petra Posedel Šimović & Zvonko Kostanjčar, 2021. "Reinforcement Learning Approaches to Optimal Market Making," Mathematics, MDPI, vol. 9(21), pages 1-22, October.
    8. Alessio Emanuele Biondo, 2020. "Information versus imitation in a real-time agent-based model of financial markets," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 15(3), pages 613-631, July.
    9. Kiran Sharma & Subhradeep Das & Anirban Chakraborti, 2017. "Global Income Inequality and Savings: A Data Science Perspective," Papers 1801.00253, arXiv.org, revised Aug 2018.
    10. Ban Zheng & François Roueff & Frédéric Abergel, 2014. "Ergodicity and scaling limit of a constrained multivariate Hawkes process," Post-Print hal-00777941, HAL.
    11. Ivan Jericevich & Patrick Chang & Tim Gebbie, 2021. "Simulation and estimation of a point-process market-model with a matching engine," Papers 2105.02211, arXiv.org, revised Aug 2021.
    12. A. Lykov & S. Muzychka & K. Vaninsky, 2012. "Investor's sentiment in multi-agent model of the continuous double auction," Papers 1208.3083, arXiv.org, revised Feb 2016.
    13. Fei Cao & Sebastien Motsch, 2021. "Derivation of wealth distributions from biased exchange of money," Papers 2105.07341, arXiv.org.
    14. Pierre Gosselin & Aïleen Lotz & Marc Wambst, 2020. "A path integral approach to business cycle models with large number of agents," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 15(4), pages 899-942, October.
    15. Chakravarty, Sugato & Harris, Fredreck H. deB. & Wood, Roger A., 2001. "Do Bid-Ask Spreads or Bid and Ask Depths Convey New Information First?," Purdue University Economics Working Papers 1149, Purdue University, Department of Economics.
    16. Pierre Gosselin & Aïleen Lotz & Marc Wambst, 2019. "A Statistical Field Approach to Capital Accumulation," Working Papers hal-02280634, HAL.
    17. Jain, Pawan & Jiang, Christine, 2014. "Predicting future price volatility: Empirical evidence from an emerging limit order market," Pacific-Basin Finance Journal, Elsevier, vol. 27(C), pages 72-93.
    18. repec:hal:wpaper:hal-00777941 is not listed on IDEAS
    19. Ben Hambly & Renyuan Xu & Huining Yang, 2021. "Recent Advances in Reinforcement Learning in Finance," Papers 2112.04553, arXiv.org, revised Feb 2023.
    20. Pierre Gosselin & Aïleen Lotz & Marc Wambst, 2017. "A Path Integral Approach to Interacting Economic Systems with Multiple Heterogeneous Agents," Working Papers hal-01549586, HAL.
    21. Cenesizoglu, Tolga & Dionne, Georges & Zhou, Xiaozhou, 2022. "Asymmetric effects of the limit order book on price dynamics," Journal of Empirical Finance, Elsevier, vol. 65(C), pages 77-98.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2305.15821. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.