IDEAS home Printed from https://ideas.repec.org/a/eee/dyncon/v144y2022ics0165188922002019.html
   My bibliography  Save this article

Reinforcement Learning Equilibrium in Limit Order Markets

Author

Listed:
  • He, Xue-Zhong
  • Lin, Shen

Abstract

This paper introduces an information-based reinforcement learning to exploit information channels to traders’ trading behavior in an equilibrium limit order market. Anticipating that informed traders are more likely to submit market buy (sell) orders when asset is significantly under (over) valued, uninformed traders tend to chase market buy (sell) orders of the informed to buy (sell). To gain from the order chasing of the uninformed, informed traders strategically submit more market buy (sell) and limit sell (buy) orders. This amplifies the order chasing behaviour of the uninformed, generating predictable trading behaviours that can improve information efficiency but reduce market liquidity. Order book information and learning can have opposite effects on order choices and endogenous liquidity provision for the informed and uninformed. Furthermore, more informed trading is beneficial, but fast trading can be harmful for market quality.

Suggested Citation

  • He, Xue-Zhong & Lin, Shen, 2022. "Reinforcement Learning Equilibrium in Limit Order Markets," Journal of Economic Dynamics and Control, Elsevier, vol. 144(C).
    • Handle: RePEc:eee:dyncon:v:144:y:2022:i:c:s0165188922002019
    DOI: 10.1016/j.jedc.2022.104497
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0165188922002019
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jedc.2022.104497?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Roberto Dieci & Xue-Zhong He, 2018. "Heterogeneous Agent Models in Finance," Research Paper Series 389, Quantitative Finance Research Centre, University of Technology, Sydney.
    2. Sebastien Pouget, 2007. "Adaptive Traders and the Design of Financial Markets," Journal of Finance, American Finance Association, vol. 62(6), pages 2835-2863, December.
    3. Chiarella, Carl & Iori, Giulia, 2009. "The impact of heterogeneous trading rules on the limit order book and order flows," Journal of Economic Dynamics and Control, Elsevier, vol. 33(3), pages 525-537.
    4. Erev, Ido & Roth, Alvin E, 1998. "Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria," American Economic Review, American Economic Association, vol. 88(4), pages 848-881, September.
    5. Biais, Bruno & Foucault, Thierry & Moinas, Sophie, 2015. "Equilibrium fast trading," Journal of Financial Economics, Elsevier, vol. 116(2), pages 292-313.
    6. Ladley, Daniel, 2020. "The high frequency trade off between speed and sophistication," Journal of Economic Dynamics and Control, Elsevier, vol. 116(C).
    7. Elise Payzan-LeNestour & Peter Bossaerts, 2015. "Learning About Unstable, Publicly Unobservable Payoffs," Review of Financial Studies, Society for Financial Studies, vol. 28(7), pages 1874-1913.
    8. Markku Kaustia & Samuli Knüpfer, 2008. "Do Investors Overweight Personal Experience? Evidence from IPO Subscriptions," Journal of Finance, American Finance Association, vol. 63(6), pages 2679-2702, December.
    9. Arthur, W.B. & Holland, J.H. & LeBaron, B. & Palmer, R. & Tayler, P., 1996. "Asset Pricing Under Endogenous Expectations in an Artificial Stock Market," Working papers 9625, Wisconsin Madison - Social Systems.
    10. Biais, Bruno & Hillion, Pierre & Spatt, Chester, 1995. "An Empirical Analysis of the Limit Order Book and the Order Flow in the Paris Bourse," Journal of Finance, American Finance Association, vol. 50(5), pages 1655-1689, December.
    11. Thierry Foucault & Johan Hombert & Ioanid Roşu, 2016. "News Trading and Speed," Journal of Finance, American Finance Association, vol. 71(1), pages 335-382, February.
    12. Yeh, Chia-Hsuan & Yang, Chun-Yi, 2010. "Examining the effectiveness of price limits in an artificial stock market," Journal of Economic Dynamics and Control, Elsevier, vol. 34(10), pages 2089-2108, October.
    13. David Easley & Marcos López de Prado & Maureen O’Hara & Zhibai Zhang & Wei Jiang, 2021. "Microstructure in the Machine Age [The risk of machine learning]," Review of Financial Studies, Society for Financial Studies, vol. 34(7), pages 3316-3363.
    14. Bongaerts, Dion & Achter, Mark Van, 2021. "Competition among liquidity providers with access to high-frequency trading technology," Journal of Financial Economics, Elsevier, vol. 140(1), pages 220-249.
    15. Theissen, Erik, 2000. "Market structure, informational efficiency and liquidity: An experimental comparison of auction and dealer markets," Journal of Financial Markets, Elsevier, vol. 3(4), pages 333-363, November.
    16. James J. Choi & David Laibson & Brigitte C. Madrian & Andrew Metrick, 2009. "Reinforcement Learning and Savings Behavior," Journal of Finance, American Finance Association, vol. 64(6), pages 2515-2534, December.
    17. Carl Chiarella & Giulia Iori, 2002. "A simulation analysis of the microstructure of double auction markets," Quantitative Finance, Taylor & Francis Journals, vol. 2(5), pages 346-353.
    18. LeBaron, Blake, 2006. "Agent-based Computational Finance," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 24, pages 1187-1233, Elsevier.
    19. Goettler, Ronald L. & Parlour, Christine A. & Rajan, Uday, 2009. "Informed traders and limit order markets," Journal of Financial Economics, Elsevier, vol. 93(1), pages 67-87, July.
    20. Lensberg, Terje & Schenk-Hoppé, Klaus Reiner & Ladley, Dan, 2015. "Costs and benefits of financial regulation: Short-selling bans and transaction taxes," Journal of Banking & Finance, Elsevier, vol. 51(C), pages 103-118.
    21. LeBaron, Blake & Arthur, W. Brian & Palmer, Richard, 1999. "Time series properties of an artificial stock market," Journal of Economic Dynamics and Control, Elsevier, vol. 23(9-10), pages 1487-1516, September.
    22. Hommes, Cars H., 2006. "Heterogeneous Agent Models in Economics and Finance," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 23, pages 1109-1186, Elsevier.
    23. David Easley & Marcos López de Prado & Maureen O’Hara & Zhibai Zhang, 2021. "Microstructure in the Machine Age," NBER Chapters, in: Big Data: Long-Term Implications for Financial Markets and Firms, pages 3316-3363, National Bureau of Economic Research, Inc.
    24. Shu-Heng Chen & Yi-Lin Hsieh, 2011. "Reinforcement Learning in Experimental Asset Markets," Eastern Economic Journal, Palgrave Macmillan;Eastern Economic Association, vol. 37(1), pages 109-133.
    25. Breckenfelder, Johannes, 2013. "Competition between high-frequency traders, and market quality," MPRA Paper 66715, University Library of Munich, Germany, revised Dec 2013.
    26. James J. Choi & David Laibson & Brigitte C. Madrian & Andrew Metrick, 2009. "Reinforcement Learning and Savings Behavior," Journal of Finance, American Finance Association, vol. 64(6), pages 2515-2534, December.
    27. Sendhil Mullainathan & Jann Spiess, 2017. "Machine Learning: An Applied Econometric Approach," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 87-106, Spring.
    28. Chiarella, Carl & He, Xue-Zhong & Pellizzari, Paolo, 2012. "A Dynamic Analysis Of The Microstructure Of Moving Average Rules In A Double Auction Market," Macroeconomic Dynamics, Cambridge University Press, vol. 16(4), pages 556-575, September.
    29. Albert J. Menkveld, 2016. "The Economics of High-Frequency Trading: Taking Stock," Annual Review of Financial Economics, Annual Reviews, vol. 8(1), pages 1-24, October.
    30. Duffy, John, 2006. "Agent-Based Models and Human Subject Experiments," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 19, pages 949-1011, Elsevier.
    31. Chakravarty Sugato & Holden Craig W., 1995. "An Integrated Model of Market and Limit Orders," Journal of Financial Intermediation, Elsevier, vol. 4(3), pages 213-241, July.
    32. Ronald L. Goettler & Christine A. Parlour & Uday Rajan, 2005. "Equilibrium in a Dynamic Limit Order Market," Journal of Finance, American Finance Association, vol. 60(5), pages 2149-2192, October.
    33. Brogaard, Jonathan & Garriott, Corey, 2019. "High-Frequency Trading Competition," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 54(4), pages 1469-1497, August.
    34. Chiarella, Carl & He, Xue-Zhong & Wei, Lijian, 2015. "Learning, information processing and order submission in limit order markets," Journal of Economic Dynamics and Control, Elsevier, vol. 61(C), pages 245-268.
    35. Arifovic, Jasmina & He, Xue-zhong & Wei, Lijian, 2022. "Machine learning and speed in high-frequency trading," Journal of Economic Dynamics and Control, Elsevier, vol. 139(C).
    36. Arifovic, Jasmina, 1996. "The Behavior of the Exchange Rate in the Genetic Algorithm and Experimental Economies," Journal of Political Economy, University of Chicago Press, vol. 104(3), pages 510-541, June.
    37. Hal R. Varian, 2014. "Big Data: New Tricks for Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 3-28, Spring.
    38. Hoffmann, Peter, 2014. "A dynamic limit order market with fast and slow traders," Journal of Financial Economics, Elsevier, vol. 113(1), pages 156-169.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Arifovic, Jasmina & He, Xue-zhong & Wei, Lijian, 2022. "Machine learning and speed in high-frequency trading," Journal of Economic Dynamics and Control, Elsevier, vol. 139(C).
    2. Yamamoto, Ryuichi, 2019. "Dynamic Predictor Selection And Order Splitting In A Limit Order Market," Macroeconomic Dynamics, Cambridge University Press, vol. 23(5), pages 1757-1792, July.
    3. Lijian Wei & Wei Zhang & Xue-Zhong He & Yongjie Zhang, 2013. "Learning and Information Dissemination in Limit Order Markets," Research Paper Series 333, Quantitative Finance Research Centre, University of Technology, Sydney.
    4. Chiarella, Carl & He, Xue-Zhong & Wei, Lijian, 2015. "Learning, information processing and order submission in limit order markets," Journal of Economic Dynamics and Control, Elsevier, vol. 61(C), pages 245-268.
    5. Roberto Dieci & Xue-Zhong He, 2018. "Heterogeneous Agent Models in Finance," Research Paper Series 389, Quantitative Finance Research Centre, University of Technology, Sydney.
    6. Lijian Wei & Xiong Xiong & Wei Zhang & Xue-Zhong He & Yongjie Zhang, 2017. "The effect of genetic algorithm learning with a classifier system in limit order markets," Published Paper Series 2017-3, Finance Discipline Group, UTS Business School, University of Technology, Sydney.
    7. Ladley, Daniel, 2020. "The high frequency trade off between speed and sophistication," Journal of Economic Dynamics and Control, Elsevier, vol. 116(C).
    8. Chen, Shu-Heng, 2012. "Varieties of agents in agent-based computational economics: A historical and an interdisciplinary perspective," Journal of Economic Dynamics and Control, Elsevier, vol. 36(1), pages 1-25.
    9. Chiarella, Carl & Ladley, Daniel, 2016. "Chasing trends at the micro-level: The effect of technical trading on order book dynamics," Journal of Banking & Finance, Elsevier, vol. 72(S), pages 119-131.
    10. Anufriev, Mikhail & Panchenko, Valentyn, 2009. "Asset prices, traders' behavior and market design," Journal of Economic Dynamics and Control, Elsevier, vol. 33(5), pages 1073-1090, May.
    11. Staccioli, Jacopo & Napoletano, Mauro, 2021. "An agent-based model of intra-day financial markets dynamics," Journal of Economic Behavior & Organization, Elsevier, vol. 182(C), pages 331-348.
    12. Zhou, Hao & Kalev, Petko S., 2019. "Algorithmic and high frequency trading in Asia-Pacific, now and the future," Pacific-Basin Finance Journal, Elsevier, vol. 53(C), pages 186-207.
    13. Vivien Lespagnol & Juliette Rouchier, 2018. "Trading Volume and Price Distortion: An Agent-Based Model with Heterogenous Knowledge of Fundamentals," Post-Print hal-02084910, HAL.
    14. Pyo, Dong-Jin, 2014. "A Multi-Factor Model of Heterogeneous Traders in a Dynamic Stock Market," Staff General Research Papers Archive 37358, Iowa State University, Department of Economics.
    15. Breedon, Francis & Chen, Louisa & Ranaldo, Angelo & Vause, Nicholas, 2023. "Judgment day: Algorithmic trading around the Swiss franc cap removal," Journal of International Economics, Elsevier, vol. 140(C).
    16. repec:hal:spmain:info:hdl:2441/5mqflt6amg8gab4rlqn6sbko4b is not listed on IDEAS
    17. Degryse, Hans & Karagiannis, Nikolaos, 2019. "Priority Rules," CEPR Discussion Papers 14127, C.E.P.R. Discussion Papers.
    18. Daniel Ladley, 2019. "The Design and Regulation of High Frequency Traders," Discussion Papers in Economics 19/02, Division of Economics, School of Business, University of Leicester.
    19. Ekinci, Cumhur & Ersan, Oğuz, 2022. "High-frequency trading and market quality: The case of a “slightly exposed” market," International Review of Financial Analysis, Elsevier, vol. 79(C).
    20. Alessio Emanuele Biondo, 2018. "Order book microstructure and policies for financial stability," Studies in Economics and Finance, Emerald Group Publishing Limited, vol. 35(1), pages 196-218, March.
    21. Troy Tassier, 2013. "Handbook of Research on Complexity, by J. Barkley Rosser, Jr. and Edward Elgar," Eastern Economic Journal, Palgrave Macmillan;Eastern Economic Association, vol. 39(1), pages 132-133.

    More about this item

    Keywords

    Limit order market; reinforcement learning; order chasing; endogenous liquidity provision; and price discovery;
    All these keywords.

    JEL classification:

    • G14 - Financial Economics - - General Financial Markets - - - Information and Market Efficiency; Event Studies; Insider Trading
    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques
    • D82 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Asymmetric and Private Information; Mechanism Design
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:dyncon:v:144:y:2022:i:c:s0165188922002019. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jedc .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.