IDEAS home Printed from https://ideas.repec.org/a/eee/dyncon/v172y2025ics0165188924001830.html
   My bibliography  Save this article

Reinforcement learning and rational expectations equilibrium in limit order markets

Author

Listed:
  • Zhou, Xuan
  • Lin, Shen
  • He, Xue-Zhong

Abstract

This paper shows that simple payoff-based reinforcement learning can help to achieve rational expectations equilibrium in limit order markets. In equilibrium, speculators mainly supply liquidity, while liquidity consumption increases in the private values of no-speculators with intrinsic motives for trade. Driven by information acquisition of the non-speculators, liquidity consumption is hump-shaped in fundamental volatility for the speculators but U-shaped for the non-speculators. In contrast, liquidity supply decreases in fundamental volatility for the speculators but is hump-shaped for the non-speculators. Unlike the informed traders who trade on asset fundamentals, the uninformed traders trade more on order book and trading information.

Suggested Citation

  • Zhou, Xuan & Lin, Shen & He, Xue-Zhong, 2025. "Reinforcement learning and rational expectations equilibrium in limit order markets," Journal of Economic Dynamics and Control, Elsevier, vol. 172(C).
    • Handle: RePEc:eee:dyncon:v:172:y:2025:i:c:s0165188924001830
    DOI: 10.1016/j.jedc.2024.104991
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0165188924001830
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jedc.2024.104991?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Roberto Dieci & Xue-Zhong He, 2018. "Heterogeneous Agent Models in Finance," Research Paper Series 389, Quantitative Finance Research Centre, University of Technology, Sydney.
    2. Chiarella, Carl & Iori, Giulia, 2009. "The impact of heterogeneous trading rules on the limit order book and order flows," Journal of Economic Dynamics and Control, Elsevier, vol. 33(3), pages 525-537.
    3. Erev, Ido & Roth, Alvin E, 1998. "Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria," American Economic Review, American Economic Association, vol. 88(4), pages 848-881, September.
    4. Ladley, Daniel, 2020. "The high frequency trade off between speed and sophistication," Journal of Economic Dynamics and Control, Elsevier, vol. 116(C).
    5. Biais, Bruno & Hillion, Pierre & Spatt, Chester, 1995. "An Empirical Analysis of the Limit Order Book and the Order Flow in the Paris Bourse," Journal of Finance, American Finance Association, vol. 50(5), pages 1655-1689, December.
    6. Arthur, W.B. & Holland, J.H. & LeBaron, B. & Palmer, R. & Tayler, P., 1996. "Asset Pricing Under Endogenous Expectations in an Artificial Stock Market," Working papers 9625, Wisconsin Madison - Social Systems.
    7. Yeh, Chia-Hsuan & Yang, Chun-Yi, 2010. "Examining the effectiveness of price limits in an artificial stock market," Journal of Economic Dynamics and Control, Elsevier, vol. 34(10), pages 2089-2108, October.
    8. Carl Chiarella & Giulia Iori, 2002. "A simulation analysis of the microstructure of double auction markets," Quantitative Finance, Taylor & Francis Journals, vol. 2(5), pages 346-353.
    9. Glosten, Lawrence R. & Milgrom, Paul R., 1985. "Bid, ask and transaction prices in a specialist market with heterogeneously informed traders," Journal of Financial Economics, Elsevier, vol. 14(1), pages 71-100, March.
    10. LeBaron, Blake, 2006. "Agent-based Computational Finance," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 24, pages 1187-1233, Elsevier.
    11. Goettler, Ronald L. & Parlour, Christine A. & Rajan, Uday, 2009. "Informed traders and limit order markets," Journal of Financial Economics, Elsevier, vol. 93(1), pages 67-87, July.
    12. Lensberg, Terje & Schenk-Hoppé, Klaus Reiner & Ladley, Dan, 2015. "Costs and benefits of financial regulation: Short-selling bans and transaction taxes," Journal of Banking & Finance, Elsevier, vol. 51(C), pages 103-118.
    13. Hommes, Cars H., 2006. "Heterogeneous Agent Models in Economics and Finance," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 23, pages 1109-1186, Elsevier.
    14. Shu-Heng Chen & Yi-Lin Hsieh, 2011. "Reinforcement Learning in Experimental Asset Markets," Eastern Economic Journal, Palgrave Macmillan;Eastern Economic Association, vol. 37(1), pages 109-133.
    15. Chiarella, Carl & He, Xue-Zhong & Pellizzari, Paolo, 2012. "A Dynamic Analysis Of The Microstructure Of Moving Average Rules In A Double Auction Market," Macroeconomic Dynamics, Cambridge University Press, vol. 16(4), pages 556-575, September.
    16. Duffy, John, 2006. "Agent-Based Models and Human Subject Experiments," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 19, pages 949-1011, Elsevier.
    17. Maureen O'Hara, 2001. "Overview: market structure issues in market liquidity," BIS Papers chapters, in: Bank for International Settlements (ed.), Market liquidity: proceedings of a workshop held at the BIS, volume 2, pages 1-8, Bank for International Settlements.
    18. Chiarella, Carl & He, Xue-Zhong & Wei, Lijian, 2015. "Learning, information processing and order submission in limit order markets," Journal of Economic Dynamics and Control, Elsevier, vol. 61(C), pages 245-268.
    19. Arifovic, Jasmina & He, Xue-zhong & Wei, Lijian, 2022. "Machine learning and speed in high-frequency trading," Journal of Economic Dynamics and Control, Elsevier, vol. 139(C).
    20. Kyle, Albert S, 1985. "Continuous Auctions and Insider Trading," Econometrica, Econometric Society, vol. 53(6), pages 1315-1335, November.
    21. He, Xue-Zhong & Lin, Shen, 2022. "Reinforcement Learning Equilibrium in Limit Order Markets," Journal of Economic Dynamics and Control, Elsevier, vol. 144(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. He, Xue-Zhong & Lin, Shen, 2022. "Reinforcement Learning Equilibrium in Limit Order Markets," Journal of Economic Dynamics and Control, Elsevier, vol. 144(C).
    2. Breckenfelder, Johannes, 2024. "Competition among high-frequency traders and market quality," Journal of Economic Dynamics and Control, Elsevier, vol. 166(C).
    3. Anufriev, Mikhail & Panchenko, Valentyn, 2009. "Asset prices, traders' behavior and market design," Journal of Economic Dynamics and Control, Elsevier, vol. 33(5), pages 1073-1090, May.
    4. Roberto Dieci & Xue-Zhong He, 2018. "Heterogeneous Agent Models in Finance," Research Paper Series 389, Quantitative Finance Research Centre, University of Technology, Sydney.
    5. Alessio Emanuele Biondo, 2019. "Order book modeling and financial stability," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 14(3), pages 469-489, September.
    6. Biondo, Alessio Emanuele, 2018. "Learning to forecast, risk aversion, and microstructural aspects of financial stability," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 12, pages 1-21.
    7. Alessio Emanuele Biondo, 2018. "Order book microstructure and policies for financial stability," Studies in Economics and Finance, Emerald Group Publishing Limited, vol. 35(1), pages 196-218, March.
    8. Biondo, Alessio Emanuele, 2017. "Learning to forecast, risk aversion, and microstructural aspects of financial stability," Economics Discussion Papers 2017-104, Kiel Institute for the World Economy (IfW Kiel).
    9. Lijian Wei & Wei Zhang & Xue-Zhong He & Yongjie Zhang, 2013. "Learning and Information Dissemination in Limit Order Markets," Research Paper Series 333, Quantitative Finance Research Centre, University of Technology, Sydney.
    10. Yamamoto, Ryuichi, 2019. "Dynamic Predictor Selection And Order Splitting In A Limit Order Market," Macroeconomic Dynamics, Cambridge University Press, vol. 23(5), pages 1757-1792, July.
    11. Staccioli, Jacopo & Napoletano, Mauro, 2021. "An agent-based model of intra-day financial markets dynamics," Journal of Economic Behavior & Organization, Elsevier, vol. 182(C), pages 331-348.
    12. Vivien Lespagnol & Juliette Rouchier, 2018. "Trading Volume and Price Distortion: An Agent-Based Model with Heterogenous Knowledge of Fundamentals," Computational Economics, Springer;Society for Computational Economics, vol. 51(4), pages 991-1020, April.
    13. Ivan Jericevich & Patrick Chang & Tim Gebbie, 2021. "Simulation and estimation of an agent-based market-model with a matching engine," Papers 2108.07806, arXiv.org, revised Aug 2021.
    14. Kun Xing & Honggang Li, 2024. "Market Ecology: Trading Strategies and Market Volatility," Computational Economics, Springer;Society for Computational Economics, vol. 64(6), pages 3333-3351, December.
    15. Arifovic, Jasmina & He, Xue-zhong & Wei, Lijian, 2022. "Machine learning and speed in high-frequency trading," Journal of Economic Dynamics and Control, Elsevier, vol. 139(C).
    16. Paolo Mazza & Mikael Petitjean, 2019. "Testing the effect of technical analysis on market quality and order book dynamics," Applied Economics, Taylor & Francis Journals, vol. 51(18), pages 1947-1976, April.
    17. Biondo, Alessio Emanuele & Mazzarino, Laura & Pluchino, Alessandro, 2024. "Trading strategies and Financial Performances: A simulation approach," International Review of Financial Analysis, Elsevier, vol. 95(PB).
    18. Ya-Chi Huang & Chueh-Yung Tsao, 2018. "Discovering Traders’ Heterogeneous Behavior in High-Frequency Financial Data," Computational Economics, Springer;Society for Computational Economics, vol. 51(4), pages 821-846, April.
    19. Gao-Feng Gu & Xiong Xiong & Hai-Chuan Xu & Wei Zhang & Yongjie Zhang & Wei Chen & Wei-Xing Zhou, 2021. "An empirical behavioral order-driven model with price limit rules," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 7(1), pages 1-24, December.
    20. Baruch, Shmuel & Panayides, Marios & Venkataraman, Kumar, 2017. "Informed trading and price discovery before corporate events," Journal of Financial Economics, Elsevier, vol. 125(3), pages 561-588.

    More about this item

    Keywords

    Limit order market; Reinforcement learning; Rational expectations; Liquidity supply and consumption;
    All these keywords.

    JEL classification:

    • G14 - Financial Economics - - General Financial Markets - - - Information and Market Efficiency; Event Studies; Insider Trading
    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques
    • D82 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Asymmetric and Private Information; Mechanism Design
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:dyncon:v:172:y:2025:i:c:s0165188924001830. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jedc .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.