IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2206.08781.html
   My bibliography  Save this paper

Reinforcement Learning for Economic Policy: A New Frontier?

Author

Listed:
  • Callum Rhys Tilbury

Abstract

Agent-based computational economics is a field with a rich academic history, yet one which has struggled to enter mainstream policy design toolboxes, plagued by the challenges associated with representing a complex and dynamic reality. The field of Reinforcement Learning (RL), too, has a rich history, and has recently been at the centre of several exponential developments. Modern RL implementations have been able to achieve unprecedented levels of sophistication, handling previously unthinkable degrees of complexity. This review surveys the historical barriers of classical agent-based techniques in economic modelling, and contemplates whether recent developments in RL can overcome any of them.

Suggested Citation

  • Callum Rhys Tilbury, 2022. "Reinforcement Learning for Economic Policy: A New Frontier?," Papers 2206.08781, arXiv.org, revised Feb 2023.
  • Handle: RePEc:arx:papers:2206.08781
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2206.08781
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Kutschinski, Erich & Uthmann, Thomas & Polani, Daniel, 2003. "Learning competitive pricing strategies by multi-agent reinforcement learning," Journal of Economic Dynamics and Control, Elsevier, vol. 27(11-12), pages 2207-2218, September.
    2. Stephan Zheng & Alexander Trott & Sunil Srinivasa & Nikhil Naik & Melvin Gruesbeck & David C. Parkes & Richard Socher, 2020. "The AI Economist: Improving Equality and Productivity with AI-Driven Tax Policies," Papers 2004.13332, arXiv.org.
    3. A G Haldane & A E Turrell, 2018. "An interdisciplinary model for macroeconomics," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 34(1-2), pages 219-251.
    4. Marco Raberto & Andrea Teglio & Silvano Cincotti, 2008. "Integrating Real and Financial Markets in an Agent-Based Economic Model: An Application to Monetary Policy Design," Computational Economics, Springer;Society for Computational Economics, vol. 32(1), pages 147-162, September.
    5. Ajay Agrawal & Joshua Gans & Avi Goldfarb, 2019. "The Economics of Artificial Intelligence: An Agenda," NBER Books, National Bureau of Economic Research, Inc, number agra-1, March.
    6. Leigh Tesfatsion & Kenneth L. Judd (ed.), 2006. "Handbook of Computational Economics," Handbook of Computational Economics, Elsevier, edition 1, volume 2, number 2.
    7. Kutschinski, Erich & Uthmann, Thomas & Polani, Daniel, 2003. "Learning competitive pricing strategies by multi-agent reinforcement learning," Journal of Economic Dynamics and Control, Elsevier, vol. 27(11), pages 2207-2218.
    8. Westerhoff, Frank & Franke, Reiner, 2012. "Agent-based models for economic policy design: Two illustrative examples," BERG Working Paper Series 88, Bamberg University, Bamberg Economic Research Group.
    9. Holland, John H & Miller, John H, 1991. "Artificial Adaptive Agents in Economic Theory," American Economic Review, American Economic Association, vol. 81(2), pages 365-371, May.
    10. Herbert Dawid & Michael Neugart, 2011. "Agent-based Models for Economic Policy Design," Eastern Economic Journal, Palgrave Macmillan;Eastern Economic Association, vol. 37(1), pages 44-50.
    11. LeBaron Blake & Winker Peter, 2008. "Introduction to the Special Issue on Agent-Based Models for Economic Policy Advice," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 228(2-3), pages 141-148, April.
    12. J. Doyne Farmer & Duncan Foley, 2009. "The economy needs agent-based modelling," Nature, Nature, vol. 460(7256), pages 685-686, August.
    13. Christophe Deissenberg & Sander van Der Hoog & Herbert Dawid, 2008. "EURACE: A Massively Parallel Agent-Based Model of the European Economy," Working Papers halshs-00339756, HAL.
    14. Amir Mosavi & Pedram Ghamisi & Yaser Faghan & Puhong Duan, 2020. "Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics," Papers 2004.01509, arXiv.org.
    15. Mosavi, Amir & Faghan, Yaser & Ghamisi, Pedram & Duan, Puhong & Ardabili, Sina Faizollahzadeh & Hassan, Salwana & Band, Shahab S., 2020. "Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics," OSF Preprints jrc58, Center for Open Science.
    16. Stefan Holm & Renato Lemm & Oliver Thees & Lorenz M. Hilty, 2016. "Enhancing Agent-Based Models with Discrete Choice Experiments," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 19(3), pages 1-3.
    17. Giorgio Fagiolo & Alessio Moneta & Paul Windrum, 2007. "A Critical Guide to Empirical Validation of Agent-Based Models in Economics: Methodologies, Procedures, and Open Problems," Computational Economics, Springer;Society for Computational Economics, vol. 30(3), pages 195-226, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Benjamin Patrick Evans & Sumitra Ganesh, 2024. "Learning and Calibrating Heterogeneous Bounded Rational Market Behaviour with Multi-Agent Reinforcement Learning," Papers 2402.00787, arXiv.org.
    2. Kshama Dwarakanath & Svitlana Vyetrenko & Peyman Tavallali & Tucker Balch, 2024. "ABIDES-Economist: Agent-Based Simulation of Economic Systems with Learning Agents," Papers 2402.09563, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Luca Riccetti & Alberto Russo & Mauro Gallegati, 2015. "An agent based decentralized matching macroeconomic model," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 10(2), pages 305-332, October.
    2. Poledna, Sebastian & Thurner, Stefan & Farmer, J. Doyne & Geanakoplos, John, 2014. "Leverage-induced systemic risk under Basle II and other credit risk policies," Journal of Banking & Finance, Elsevier, vol. 42(C), pages 199-212.
    3. Gerard Ballot & Antoine Mandel & Annick Vignes, 2015. "Agent-based modeling and economic theory: where do we stand?," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 10(2), pages 199-220, October.
    4. Tommaso Ciarli & Andre' Lorentz & Maria Savona & Marco Valente, 2012. "The role of technology, organisation, and demand in growth and income distribution," LEM Papers Series 2012/06, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    5. Bence Mérõ, 2019. "Novel Modelling of the Operation of the Financial Intermediary System – Agent-based Macro Models," Financial and Economic Review, Magyar Nemzeti Bank (Central Bank of Hungary), vol. 18(3), pages 83-113.
    6. Giorgio Fagiolo & Andrea Roventini, 2012. "Macroeconomic Policy in DSGE and Agent-Based Models," Revue de l'OFCE, Presses de Sciences-Po, vol. 0(5), pages 67-116.
    7. Hafner, Sarah & Anger-Kraavi, Annela & Monasterolo, Irene & Jones, Aled, 2020. "Emergence of New Economics Energy Transition Models: A Review," Ecological Economics, Elsevier, vol. 177(C).
    8. Gennaro Catapano & Francesco Franceschi & Valentina Michelangeli & Michele Loberto, 2021. "Macroprudential Policy Analysis via an Agent Based Model of the Real Estate Sector," Temi di discussione (Economic working papers) 1338, Bank of Italy, Economic Research and International Relations Area.
    9. Francesco Lamperti & Giovanni Dosi & Mauro Napoletano & Andrea Roventini & Alessandro Sapio, 2018. "And then he wasn't a she : Climate change and green transitions in an agent-based integrated assessment model," Working Papers hal-03443464, HAL.
    10. Cincotti, Silvano & Raberto, Marco & Teglio, Andrea, 2010. "Credit money and macroeconomic instability in the agent-based model and simulator Eurace," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 4, pages 1-32.
    11. Tommaso Ciarli & André Lorentz & Marco Valente & Maria Savona, 2019. "Structural changes and growth regimes," Journal of Evolutionary Economics, Springer, vol. 29(1), pages 119-176, March.
    12. Guerini, Mattia & Napoletano, Mauro & Roventini, Andrea, 2018. "No man is an Island: The impact of heterogeneity and local interactions on macroeconomic dynamics," Economic Modelling, Elsevier, vol. 68(C), pages 82-95.
    13. Ricetti, Luca & Russo, Alberto & Gallegati, Mauro, 2013. "Unemployment benefits and financial leverage in an agent based macroeconomic model," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 7, pages 1-44.
    14. Furtado, Bernardo Alves & Eberhardt, Isaque Daniel Rocha, 2015. "Modelo espacial simples da economia: uma proposta teórico-metodológica [A simple spatial economic model: a proposal]," MPRA Paper 67005, University Library of Munich, Germany.
    15. Sander van der Hoog, 2017. "Deep Learning in (and of) Agent-Based Models: A Prospectus," Papers 1706.06302, arXiv.org.
    16. G. Fagiolo & A. Roventini, 2009. "On the Scientific Status of Economic Policy: A Tale of Alternative Paradigms," Voprosy Ekonomiki, NP Voprosy Ekonomiki, issue 6.
    17. Carlos M. Fernández-Márquez & Matías Fuentes & Juan José Martínez & Francisco J. Vázquez, 2021. "Productivity and unemployment: an ABM approach," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 16(1), pages 133-151, January.
    18. De Grauwe, Paul & Ji, Yuemei, 2017. "Structural Reforms and Monetary Policies in a Behavioural Macroeconomic Model," CEPR Discussion Papers 12336, C.E.P.R. Discussion Papers.
    19. Kukacka, Jiri & Barunik, Jozef, 2017. "Estimation of financial agent-based models with simulated maximum likelihood," Journal of Economic Dynamics and Control, Elsevier, vol. 85(C), pages 21-45.
    20. Eric Brouillat & Maïder Saint Jean, 2020. "Mind the gap: Investigating the impact of implementation gaps on cleaner technology transition," Post-Print hal-03490256, HAL.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2206.08781. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.