IDEAS home Printed from https://ideas.repec.org/p/boe/boeewp/023264.html

Deep reinforcement learning in a monetary model

Author

Listed:
  • Mingli Chen

    (University of Oxford)

  • Rama Cont

    (University of Warwick)

  • Andreas Joseph

    (Bank of England)

  • Michael Kumhof

    (Bank of England)

  • Xinlei Pan

    (University of California (Berkeley))

  • Wei Xiong

    (University of Oxford)

  • Xuan Zhou

    (Reserve Bank of Australia)

Abstract

We propose deep reinforcement learning (DRL) as a general approach to bounded rationality in dynamic stochastic general equilibrium (DSGE) models. Agents are represented by deep artificial neural networks and learn to maximise their intertemporal objective function by interacting with an a priori unknown environment. Applying this approach to a model from the adaptive learning literature, DRL agents can learn all equilibria irrespective of local stability properties. However, learning is slow and may be unstable without the imposition of early stopping criteria. These findings can have implications for the use and interpretation of DRL agents and of DSGE models more generally.

Suggested Citation

  • Mingli Chen & Rama Cont & Andreas Joseph & Michael Kumhof & Xinlei Pan & Wei Xiong & Xuan Zhou, 2025. "Deep reinforcement learning in a monetary model," Bank of England Staff Working Paper series 1142, Bank of England.
    • Handle: RePEc:boe:boeewp:023264
    as

    Download full text from publisher

    File URL: https://www.bankofengland.co.uk/-/media/boe/files/working-paper/2025/deep-reinforcement-learning-in-a-monetary-model.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Douglas Heaven, 2019. "Why deep-learning AIs are so easy to fool," Nature, Nature, vol. 574(7777), pages 163-166, October.
    2. George William Evans, 2001. "Expectations in Macroeconomics Adaptive versus Eductive Learning," Revue économique, Presses de Sciences-Po, vol. 52(3), pages 573-582.
    3. Blanchard, Olivier Jean & Kahn, Charles M, 1980. "The Solution of Linear Difference Models under Rational Expectations," Econometrica, Econometric Society, vol. 48(5), pages 1305-1311, July.
    4. Alan Kirman, 1993. "Ants, Rationality, and Recruitment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 108(1), pages 137-156.
    5. Sims, Christopher A., 2010. "Rational Inattention and Monetary Economics," Handbook of Monetary Economics, in: Benjamin M. Friedman & Michael Woodford (ed.), Handbook of Monetary Economics, edition 1, volume 3, chapter 4, pages 155-181, Elsevier.
    6. Stefano Eusepi & Bruce Preston, 2018. "The Science of Monetary Policy: An Imperfect Knowledge Perspective," Journal of Economic Literature, American Economic Association, vol. 56(1), pages 3-59, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. is not listed on IDEAS
    2. Rui & Shi, 2021. "Can an AI agent hit a moving target?," Papers 2110.02474, arXiv.org, revised Oct 2022.
    3. Heyang Ma & Qirui Mi & Qipeng Yang & Zijun Fan & Bo Li & Haifeng Zhang, 2025. "Think, Speak, Decide: Language-Augmented Multi-Agent Reinforcement Learning for Economic Decision-Making," Papers 2511.12876, arXiv.org, revised Mar 2026.
    4. Artem Kuriksha, 2021. "An Economy of Neural Networks: Learning from Heterogeneous Experiences," Papers 2110.11582, arXiv.org.
    5. Rui (Aruhan) Shi, 2021. "Learning from Zero: How to Make Consumption-Saving Decisions in a Stochastic Environment with an AI Algorithm," CESifo Working Paper Series 9255, CESifo.
    6. Yang, Yucheng & Wang, Chiyuan & Schaab, Andreas & Moll, Benjamin, 2025. "Structural Reinforcement Learning for Heterogeneous Agent Macroeconomics," CEPR Discussion Papers 20980, Centre for Economic Policy Research.
    7. Fernández-Villaverde, Jesús & Nuño, Galo & Perla, Jesse, 2024. "Taming the Curse of Dimensionality: Quantitative Economics with Deep Learning," CEPR Discussion Papers 19636, Centre for Economic Policy Research.
    8. Benjamin Moll, 2025. "The Trouble with Rational Expectations in Heterogeneous Agent Models: A Challenge for Macroeconomics," Papers 2508.20571, arXiv.org.
    9. Qirui Mi & Zhiyu Zhao & Chengdong Ma & Siyu Xia & Yan Song & Mengyue Yang & Jun Wang & Haifeng Zhang, 2024. "Learning Macroeconomic Policies through Dynamic Stackelberg Mean-Field Games," Papers 2403.12093, arXiv.org, revised Jun 2025.
    10. Qirui Mi & Qipeng Yang & Zijun Fan & Wentian Fan & Heyang Ma & Chengdong Ma & Siyu Xia & Bo An & Jun Wang & Haifeng Zhang, 2025. "EconGym: A Scalable AI Testbed with Diverse Economic Tasks," Papers 2506.12110, arXiv.org.
    11. Rui & Shi, 2021. "Learning from zero: how to make consumption-saving decisions in a stochastic environment with an AI algorithm," Papers 2105.10099, arXiv.org, revised Feb 2022.
    12. Hinterlang, Natascha & Tänzer, Alina, 2021. "Optimal monetary policy using reinforcement learning," Discussion Papers 51/2021, Deutsche Bundesbank.
    13. Michael Curry & Alexander Trott & Soham Phade & Yu Bai & Stephan Zheng, 2022. "Analyzing Micro-Founded General Equilibrium Models with Many Agents using Deep Reinforcement Learning," Papers 2201.01163, arXiv.org, revised Feb 2022.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jean-Paul L’Huillier & Sanjay R Singh & Donghoon Yoo, 2024. "Incorporating Diagnostic Expectations into the New Keynesian Framework," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 91(5), pages 3013-3046.
    2. Katsuyuki Shibayama, 2026. "The Duality of Dynamics: Real-Time Indeterminacy and Notional-Time Instability," Studies in Economics 2603, School of Economics, University of Kent.
    3. Hommes, Cars, 2018. "Behavioral & experimental macroeconomics and policy analysis: a complex systems approach," Working Paper Series 2201, European Central Bank.
    4. Anufriev, Mikhail & Duffy, John & Panchenko, Valentyn, 2022. "Learning in two-dimensional beauty contest games: Theory and experimental evidence," Journal of Economic Theory, Elsevier, vol. 201(C).
    5. Poledna, Sebastian & Miess, Michael Gregor & Hommes, Cars & Rabitsch, Katrin, 2023. "Economic forecasting with an agent-based model," European Economic Review, Elsevier, vol. 151(C).
    6. Grimaud, Alex & Salle, Isabelle & Vermandel, Gauthier, 2025. "A Dynare toolbox for social learning expectations," Journal of Economic Dynamics and Control, Elsevier, vol. 172(C).
    7. Warne, Anders, 2023. "DSGE model forecasting: rational expectations vs. adaptive learning," Working Paper Series 2768, European Central Bank.
    8. Chatelain, Jean-Bernard & Ralf, Kirsten, 2021. "Hopf Bifurcation From New-Keynesian Taylor Rule To Ramsey Optimal Policy," Macroeconomic Dynamics, Cambridge University Press, vol. 25(8), pages 2204-2236, December.
    9. Cole, Stephen J., 2020. "The influence of learning and price-level targeting on central bank forward guidance," Journal of Macroeconomics, Elsevier, vol. 65(C).
    10. Colasante, Annarita & Palestrini, Antonio & Russo, Alberto & Gallegati, Mauro, 2017. "Adaptive expectations versus rational expectations: Evidence from the lab," International Journal of Forecasting, Elsevier, vol. 33(4), pages 988-1006.
    11. Pfajfar, Damjan & Santoro, Emiliano, 2014. "Credit Market Distortions, Asset Prices And Monetary Policy," Macroeconomic Dynamics, Cambridge University Press, vol. 18(3), pages 631-650, April.
    12. Mitsuhiro Okano, 2026. "Equilibrium Selection and Closure in DSGE Solution Methods," Papers 2601.19329, arXiv.org, revised Sep 2026.
    13. Goy, Gavin & Hommes, Cars & Mavromatis, Kostas, 2022. "Forward guidance and the role of central bank credibility under heterogeneous beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 1240-1274.
    14. Ashwin, Julian & Beaudry, Paul & Ellison, Martin, 2025. "Neural network learning for nonlinear economies," Journal of Monetary Economics, Elsevier, vol. 149(C).
    15. García-Cicco, Javier, 2022. "Alternative monetary-policy instruments and limited credibility: An exploration," Latin American Journal of Central Banking (previously Monetaria), Elsevier, vol. 3(1).
    16. André Marine Charlotte & Dai Meixing, 2018. "Learning, robust monetary policy and the merit of precaution," The B.E. Journal of Macroeconomics, De Gruyter, vol. 18(2), pages 1-20, June.
    17. Elton Beqiraj & Giovanni Di Bartolomeo & Marco Di Pietro & Carolina Serpieri, 2020. "Bounded rationality and heterogeneous expectations: Euler versus anticipated-utility approach," Journal of Economics, Springer, vol. 130(3), pages 249-273, August.
    18. Gauti B. Eggertsson & Sergei K. Egiev, 2025. "Liquidity Traps: A Unified Theory of the Great Depression and the Great Recession," Journal of Economic Literature, American Economic Association, vol. 63(4), pages 1424-1551, December.
    19. Marine Charlotte André & Meixing Dai, 2017. "Can inflation contract discipline central bankers when agents are learning?," Working Papers of BETA 2017-25, Bureau d'Economie Théorique et Appliquée, UDS, Strasbourg.
    20. Lindé, J. & Smets, F. & Wouters, R., 2016. "Challenges for Central Banks’ Macro Models," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 2185-2262, Elsevier.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • E52 - Macroeconomics and Monetary Economics - - Monetary Policy, Central Banking, and the Supply of Money and Credit - - - Monetary Policy
    • E62 - Macroeconomics and Monetary Economics - - Macroeconomic Policy, Macroeconomic Aspects of Public Finance, and General Outlook - - - Fiscal Policy; Modern Monetary Theory

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:boe:boeewp:023264. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Research (email available below). General contact details of provider: https://edirc.repec.org/data/boegvuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.