IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2303.04833.html
   My bibliography  Save this paper

Finding Regularized Competitive Equilibria of Heterogeneous Agent Macroeconomic Models with Reinforcement Learning

Author

Listed:
  • Ruitu Xu
  • Yifei Min
  • Tianhao Wang
  • Zhaoran Wang
  • Michael I. Jordan
  • Zhuoran Yang

Abstract

We study a heterogeneous agent macroeconomic model with an infinite number of households and firms competing in a labor market. Each household earns income and engages in consumption at each time step while aiming to maximize a concave utility subject to the underlying market conditions. The households aim to find the optimal saving strategy that maximizes their discounted cumulative utility given the market condition, while the firms determine the market conditions through maximizing corporate profit based on the household population behavior. The model captures a wide range of applications in macroeconomic studies, and we propose a data-driven reinforcement learning framework that finds the regularized competitive equilibrium of the model. The proposed algorithm enjoys theoretical guarantees in converging to the equilibrium of the market at a sub-linear rate.

Suggested Citation

  • Ruitu Xu & Yifei Min & Tianhao Wang & Zhaoran Wang & Michael I. Jordan & Zhuoran Yang, 2023. "Finding Regularized Competitive Equilibria of Heterogeneous Agent Macroeconomic Models with Reinforcement Learning," Papers 2303.04833, arXiv.org.
    • Handle: RePEc:arx:papers:2303.04833
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2303.04833
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Rahul Mazumder & Arkopal Choudhury & Garud Iyengar & Bodhisattva Sen, 2019. "A Computational Framework for Multivariate Convex Regression and Its Variants," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(525), pages 318-331, January.
    2. Lepetyuk, Vadym & Maliar, Lilia & Maliar, Serguei, 2020. "When the U.S. catches a cold, Canada sneezes: A lower-bound tale told by deep learning," Journal of Economic Dynamics and Control, Elsevier, vol. 117(C).
    3. Richard W. Blundell & Martin Browning & Ian A. Crawford, 2003. "Nonparametric Engel Curves and Revealed Preference," Econometrica, Econometric Society, vol. 71(1), pages 205-240, January.
    4. Maliar, Lilia & Maliar, Serguei, 2022. "Deep learning classification: Modeling discrete labor choice," Journal of Economic Dynamics and Control, Elsevier, vol. 135(C).
    5. S. Rao Aiyagari, 1994. "Uninsured Idiosyncratic Risk and Aggregate Saving," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 109(3), pages 659-684.
    6. Jesús Fernández‐Villaverde & Samuel Hurtado & Galo Nuño, 2023. "Financial Frictions and the Wealth Distribution," Econometrica, Econometric Society, vol. 91(3), pages 869-901, May.
    7. Yves Achdou & Jiequn Han & Jean-Michel Lasry & Pierre-Louis Lionse & Benjamin Moll, 2022. "Income and Wealth Distribution in Macroeconomics: A Continuous-Time Approach [On the Existence and Uniqueness of Stationary Equilibrium in Bewley Economies with Production]," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 89(1), pages 45-86.
    8. Eunji Lim & Peter W. Glynn, 2012. "Consistency of Multidimensional Convex Regression," Operations Research, INFORMS, vol. 60(1), pages 196-208, February.
    9. Edward C. Prescott & Rajnish Mehra, 2005. "Recursive Competitive Equilibrium: The Case Of Homogeneous Households," World Scientific Book Chapters, in: Sudipto Bhattacharya & George M Constantinides (ed.), Theory Of Valuation, chapter 11, pages 357-371, World Scientific Publishing Co. Pte. Ltd..
    10. Huggett, Mark, 1993. "The risk-free rate in heterogeneous-agent incomplete-insurance economies," Journal of Economic Dynamics and Control, Elsevier, vol. 17(5-6), pages 953-969.
    11. Hommes, Cars H., 2006. "Heterogeneous Agent Models in Economics and Finance," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 23, pages 1109-1186, Elsevier.
    12. Achdou, Yves & Han, Jiequn & Lasry, Jean Michel & Lions, Pierre Louis & Moll, Ben, 2022. "Income and wealth distribution in macroeconomics: a continuous-time approach," LSE Research Online Documents on Economics 107422, London School of Economics and Political Science, LSE Library.
    13. Moritz Kuhn, 2013. "Recursive Equilibria In An Aiyagari‐Style Economy With Permanent Income Shocks," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 54(3), pages 807-835, August.
    14. Bar Light, 2020. "Uniqueness of equilibrium in a Bewley–Aiyagari model," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 69(2), pages 435-450, March.
    15. Raphael Fonteneau & Susan Murphy & Louis Wehenkel & Damien Ernst, 2013. "Batch mode reinforcement learning based on the synthesis of artificial trajectories," Annals of Operations Research, Springer, vol. 208(1), pages 383-416, September.
    16. James E. Smith & Kevin F. McCardle, 2002. "Structural Properties of Stochastic Dynamic Programs," Operations Research, INFORMS, vol. 50(5), pages 796-809, October.
    17. Moritz Kuhn, 2013. "Recursive Equilibria In An Aiyagari‐Style Economy With Permanent Income Shocks," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 54, pages 807-835, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Marcello D'Amato & Christian Di Pietro & Marco M. Sorge, 2023. "Left and Right: A Tale of Two Tails of the Wealth Distribution," CSEF Working Papers 691, Centre for Studies in Economics and Finance (CSEF), University of Naples, Italy.
    2. Thomas J. Sargent & John Stachurski, 2024. "Dynamic Programming: Finite States," Papers 2401.10473, arXiv.org.
    3. Andrew Lyasoff, 2023. "Self-Aware Transport of Economic Agents," Papers 2303.12567, arXiv.org, revised Jan 2024.
    4. Frederick Ploeg, 2023. "Fiscal Costs of Climate Policies: Role of Tax, Political, and Behavioural Distortions," De Economist, Springer, vol. 171(2), pages 119-137, June.
    5. Moritz Kuhn & Gašper Ploj, 2020. "Job Stability, Earnings Dynamics, and Life-Cycle Savings," CESifo Working Paper Series 8710, CESifo.
    6. Zongwu Cai & Hongwei Mei & Rui Wang, 2024. "Model Specification Tests of Heterogenous Agent Models with Aggregate Shocks under Partial Information," WORKING PAPERS SERIES IN THEORETICAL AND APPLIED ECONOMICS 202405, University of Kansas, Department of Economics, revised Feb 2024.
    7. Keyvan Eslami & Tom Phelan, 2023. "The Art of Temporal Approximation An Investigation into Numerical Solutions to Discrete and Continuous-Time Problems in Economics," Working Papers 23-10, Federal Reserve Bank of Cleveland.
    8. Dirk Krueger & Harald Uhlig, 2024. "Neoclassical Growth with Limited Commitment," PIER Working Paper Archive 22-023, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
    9. Dmitriy Sergeyev & Chen Lian & Yuriy Gorodnichenko, 2023. "The Economics of Financial Stress," NBER Working Papers 31285, National Bureau of Economic Research, Inc.
    10. Victor Duarte & Diogo Duarte & Dejanir H. Silva, 2024. "Machine Learning for Continuous-Time Finance," CESifo Working Paper Series 10909, CESifo.
    11. John Stachurski, 2009. "Economic Dynamics: Theory and Computation," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262012774, December.
    12. Shenghao Zhu, 2020. "Existence Of Stationary Equilibrium In An Incomplete‐Market Model With Endogenous Labor Supply," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 61(3), pages 1115-1138, August.
    13. Chipeniuk, Karsten O. & Katz, Nets Hawk & Walker, Todd B., 2022. "Households, auctioneers, and aggregation," European Economic Review, Elsevier, vol. 141(C).
    14. Athreya, Kartik B., 2014. "Big Ideas in Macroeconomics: A Nontechnical View," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262019736, December.
    15. Jesús Fernández-Villaverde & Joël Marbet & Galo Nuño & Omar Rachedi, 2023. "Inequality and the Zero Lower Bound," NBER Working Papers 31282, National Bureau of Economic Research, Inc.
    16. Tom Krebs, 2006. "Recursive equilibrium in endogenous growth models with incomplete markets," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 29(3), pages 505-523, November.
    17. David Staines, 2023. "Stochastic Equilibrium the Lucas Critique and Keynesian Economics," Papers 2312.16214, arXiv.org.
    18. Alexandre Gaillard & Christian Hellwig & Philipp Wangner & Nicolas Werquin, 2023. "Consumption, Wealth, and Income Inequality: A Tale of Tails," Working Paper Series WP 2023-43, Federal Reserve Bank of Chicago.
    19. Li, Huiyu & Stachurski, John, 2014. "Solving the income fluctuation problem with unbounded rewards," Journal of Economic Dynamics and Control, Elsevier, vol. 45(C), pages 353-365.
    20. Soyoung Lee, 2023. "The Macroeconomic Effects of Debt Relief Policies During Recessions," Staff Working Papers 23-48, Bank of Canada.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2303.04833. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.