IDEAS home Printed from
   My bibliography  Save this paper

Building an Artificial Stock Market Populated by Reinforcement-Learning Agents


  • Tomas Ramanauskas

    () (Bank of Lithuania)

  • Aleksandras Vytautas Rutkauskas

    (Vilnius Gediminas Technical University)


In this paper we propose an artificial stock market model based on interaction of heterogeneous agents whose forward-looking behaviour is driven by the reinforcement learning algorithm combined with some evolutionary selection mechanism. We use the model for the analysis of market self-regulation abilities, market efficiency and determinants of emergent properties of the financial market. Distinctive and novel features of the model include strong emphasis on the economic content of individual decision making, application of the Q-learning algorithm for driving individual behaviour, and rich market setup.

Suggested Citation

  • Tomas Ramanauskas & Aleksandras Vytautas Rutkauskas, 2009. "Building an Artificial Stock Market Populated by Reinforcement-Learning Agents," Bank of Lithuania Working Paper Series 6, Bank of Lithuania.
  • Handle: RePEc:lie:wpaper:6

    Download full text from publisher

    File URL:
    File Function: Full text
    Download Restriction: no

    References listed on IDEAS

    1. Orphanides, Athanasios & van Norden, Simon, 2005. "The Reliability of Inflation Forecasts Based on Output Gap Estimates in Real Time," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 37(3), pages 583-601, June.
    2. Jordi Galí & Mark Gertler, 2007. "Macroeconomic Modeling for Monetary Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 21(4), pages 25-46, Fall.
    3. Jordi Galí & Mark Gertler & J. David López-Salido, 2007. "Markups, Gaps, and the Welfare Costs of Business Fluctuations," The Review of Economics and Statistics, MIT Press, vol. 89(1), pages 44-59, November.
    4. Warne, Anders & Coenen, Günter & Christoffel, Kai, 2008. "The new area-wide model of the euro area: a micro-founded open-economy model for forecasting and policy analysis," Working Paper Series 944, European Central Bank.
    5. Gregory de Walque & Frank Smets & Raf Wouters, 2006. "Price Shocks in General Equilibrium: Alternative Specifications," CESifo Economic Studies, CESifo, vol. 52(1), pages 153-176, March.
    6. Adolfson, Malin & Laseen, Stefan & Linde, Jesper & Villani, Mattias, 2007. "Bayesian estimation of an open economy DSGE model with incomplete pass-through," Journal of International Economics, Elsevier, vol. 72(2), pages 481-511, July.
    7. Raf Wouters & Frank Smets, 2005. "Comparing shocks and frictions in US and euro area business cycles: a Bayesian DSGE Approach," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(2), pages 161-183.
    8. Paul Fenton & Stephen Murchison, 2006. "ToTEM: The Bank of Canada's New Projection and Policy-Analysis Model," Bank of Canada Review, Bank of Canada, vol. 2006(Autumn), pages 5-18.
    9. Neiss, Katharine S. & Nelson, Edward, 2003. "The Real-Interest-Rate Gap As An Inflation Indicator," Macroeconomic Dynamics, Cambridge University Press, vol. 7(02), pages 239-262, April.
    10. Frank Smets & Rafael Wouters, 2007. "Shocks and Frictions in US Business Cycles: A Bayesian DSGE Approach," American Economic Review, American Economic Association, vol. 97(3), pages 586-606, June.
    11. Stephen Murchison & Andrew Rennison, 2006. "ToTEM: The Bank of Canada's New Quarterly Projection Model," Technical Reports 97, Bank of Canada.
    Full references (including those not matched with items on IDEAS)

    More about this item


    agent-based financial modelling; artificial stock market; complex dynamical system; emergent properties; market efficiency; agent heterogeneity; reinforcement learning;

    JEL classification:

    • G10 - Financial Economics - - General Financial Markets - - - General (includes Measurement and Data)
    • G11 - Financial Economics - - General Financial Markets - - - Portfolio Choice; Investment Decisions
    • G14 - Financial Economics - - General Financial Markets - - - Information and Market Efficiency; Event Studies; Insider Trading

    NEP fields

    This paper has been announced in the following NEP Reports:


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:lie:wpaper:6. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Mihnea Constantinescu). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.