IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2403.12093.html
   My bibliography  Save this paper

Learning Macroeconomic Policies through Dynamic Stackelberg Mean-Field Games

Author

Listed:
  • Qirui Mi
  • Zhiyu Zhao
  • Chengdong Ma
  • Siyu Xia
  • Yan Song
  • Mengyue Yang
  • Jun Wang
  • Haifeng Zhang

Abstract

Macroeconomic outcomes emerge from individuals' decisions, making it essential to model how agents interact with macro policy via consumption, investment, and labor choices. We formulate this as a dynamic Stackelberg game: the government (leader) sets policies, and agents (followers) respond by optimizing their behavior over time. Unlike static models, this dynamic formulation captures temporal dependencies and strategic feedback critical to policy design. However, as the number of agents increases, explicitly simulating all agent-agent and agent-government interactions becomes computationally infeasible. To address this, we propose the Dynamic Stackelberg Mean Field Game (DSMFG) framework, which approximates these complex interactions via agent-population and government-population couplings. This approximation preserves individual-level feedback while ensuring scalability, enabling DSMFG to jointly model three core features of real-world policymaking: dynamic feedback, asymmetry, and large scale. We further introduce Stackelberg Mean Field Reinforcement Learning (SMFRL), a data-driven algorithm that learns the leader's optimal policies while maintaining personalized responses for individual agents. Empirically, we validate our approach in a large-scale simulated economy, where it scales to 1,000 agents (vs. 100 in prior work) and achieves a fourfold increase in GDP over classical economic methods and a nineteenfold improvement over the static 2022 U.S. federal income tax policy.

Suggested Citation

  • Qirui Mi & Zhiyu Zhao & Chengdong Ma & Siyu Xia & Yan Song & Mengyue Yang & Jun Wang & Haifeng Zhang, 2024. "Learning Macroeconomic Policies through Dynamic Stackelberg Mean-Field Games," Papers 2403.12093, arXiv.org, revised Jun 2025.
  • Handle: RePEc:arx:papers:2403.12093
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2403.12093
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Sungbae An & Frank Schorfheide, 2007. "Bayesian Analysis of DSGE Models—Rejoinder," Econometric Reviews, Taylor & Francis Journals, vol. 26(2-4), pages 211-219.
    2. Steven Campbell & Yichao Chen & Arvind Shrivats & Sebastian Jaimungal, 2021. "Deep Learning for Principal-Agent Mean Field Games," Papers 2110.01127, arXiv.org.
    3. Olivier Blanchard & Jordi Galí, 2007. "Real Wage Rigidities and the New Keynesian Model," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 39(s1), pages 35-65, February.
    4. Xavier Gabaix, 2020. "A Behavioral New Keynesian Model," American Economic Review, American Economic Association, vol. 110(8), pages 2271-2327, August.
    5. Persson, Torsten & Tabellini, Guido, 1999. "Political economics and macroeconomic policy," Handbook of Macroeconomics, in: J. B. Taylor & M. Woodford (ed.), Handbook of Macroeconomics, edition 1, volume 1, chapter 22, pages 1397-1482, Elsevier.
    6. Lee, Kevin & Pesaran, M Hashem & Smith, Ron, 1997. "Growth and Convergence in Multi-country Empirical Stochastic Solow Model," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 12(4), pages 357-392, July-Aug..
    7. Sungbae An & Frank Schorfheide, 2007. "Bayesian Analysis of DSGE Models," Econometric Reviews, Taylor & Francis Journals, vol. 26(2-4), pages 113-172.
    8. Fatih Ozhamaratli & Paolo Barucca, 2022. "Deep Reinforcement Learning for Optimal Investment and Saving Strategy Selection in Heterogeneous Profiles: Intelligent Agents working towards retirement," Papers 2206.05835, arXiv.org.
    9. Amitava Krishna Dutt, 2006. "Aggregate Demand, Aggregate Supply and Economic Growth," International Review of Applied Economics, Taylor & Francis Journals, vol. 20(3), pages 319-336.
    10. Jordi Galí, 1992. "How Well Does The IS-LM Model Fit Postwar U. S. Data?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 107(2), pages 709-738.
    11. Frank Smets & Rafael Wouters, 2007. "Shocks and Frictions in US Business Cycles: A Bayesian DSGE Approach," American Economic Review, American Economic Association, vol. 97(3), pages 586-606, June.
    12. Mingli Chen & Andreas Joseph & Michael Kumhof & Xinlei Pan & Xuan Zhou, 2021. "Deep Reinforcement Learning in a Monetary Model," Papers 2104.09368, arXiv.org, revised Jan 2023.
    13. Emmanuel Saez, 2001. "Using Elasticities to Derive Optimal Income Tax Rates," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 68(1), pages 205-229.
    14. Michael Curry & Alexander Trott & Soham Phade & Yu Bai & Stephan Zheng, 2022. "Analyzing Micro-Founded General Equilibrium Models with Many Agents using Deep Reinforcement Learning," Papers 2201.01163, arXiv.org, revised Feb 2022.
    15. Tohid Atashbar & Rui Aruhan Shi, 2023. "AI and Macroeconomic Modeling: Deep Reinforcement Learning in an RBC model," IMF Working Papers 2023/040, International Monetary Fund.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Benchimol, Jonathan & Bounader, Lahcen, 2023. "Optimal monetary policy under bounded rationality," Journal of Financial Stability, Elsevier, vol. 67(C).
    2. Sala, Luca & Söderström, Ulf & Trigari, Antonella, 2008. "Monetary policy under uncertainty in an estimated model with labor market frictions," Journal of Monetary Economics, Elsevier, vol. 55(5), pages 983-1006, July.
    3. Giovanni Melina & Stefania Villa, 2018. "Leaning Against Windy Bank Lending," Economic Inquiry, Western Economic Association International, vol. 56(1), pages 460-482, January.
    4. Pablo Cuba-Borda & Sanjay R. Singh, 2024. "Understanding Persistent ZLB: Theory and Assessment," American Economic Journal: Macroeconomics, American Economic Association, vol. 16(3), pages 389-416, July.
    5. Luca Benati, 2008. "Investigating Inflation Persistence Across Monetary Regimes," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 123(3), pages 1005-1060.
    6. Mariusz Górajski & Zbigniew Kuchta, 2022. "Which hallmarks of optimal monetary policy rules matter in Poland? A stochastic dominance approach," Bank i Kredyt, Narodowy Bank Polski, vol. 53(2), pages 149-182.
    7. Loberto, Michele & Perricone, Chiara, 2017. "Does trend inflation make a difference?," Economic Modelling, Elsevier, vol. 61(C), pages 351-375.
    8. Zoundi, Zakaria, 2024. "Wells or Welfare? Macroeconomic implications of the Canadian oil subsidy," Economic Modelling, Elsevier, vol. 139(C).
    9. Ewen Gallic & Jean-Christophe Poutineau & Gauthier Vermandel, 2017. "L’impact de la crise financière sur la performance de la politique monétaire conventionnelle de la zone euro," Revue économique, Presses de Sciences-Po, vol. 68(HS1), pages 63-86.
    10. repec:zbw:bofrdp:2016_016 is not listed on IDEAS
    11. Kocięcki, Andrzej & Kolasa, Marcin, 2023. "A solution to the global identification problem in DSGE models," Journal of Econometrics, Elsevier, vol. 236(2).
    12. Mădălin Viziniuc, 2017. "Potential Gains from Cooperation Between Monetary and Macroprudential Policies: The Case of an Emerging Economy," Eastern European Economics, Taylor & Francis Journals, vol. 55(5), pages 420-452, September.
    13. Gelfer, Sacha & Gibbs, Christopher G., 2023. "Measuring the effects of large-scale asset purchases: The role of international financial markets and the financial accelerator," Journal of International Money and Finance, Elsevier, vol. 131(C).
    14. Sarah Mouabbi & Jean‐Guillaume Sahuc, 2019. "Evaluating the Macroeconomic Effects of the ECB's Unconventional Monetary Policies," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 51(4), pages 831-858, June.
    15. Del Negro, Marco & Schorfheide, Frank, 2008. "Forming priors for DSGE models (and how it affects the assessment of nominal rigidities)," Journal of Monetary Economics, Elsevier, vol. 55(7), pages 1191-1208, October.
    16. Jean-Christophe Poutineau & Gauthier Vermandel, 2015. "Quelle prise en compte des caractéristiques nationales dans les mesures macroprudentielles en zone euro ?," Revue française d'économie, Presses de Sciences-Po, vol. 0(3), pages 95-141.
    17. Yao, Fang, 2009. "Real and nominal rigidities in price setting: A bayesian analysis using aggregate data," SFB 649 Discussion Papers 2009-057, Humboldt University Berlin, Collaborative Research Center 649: Economic Risk.
    18. Eric Jondeau & Gregory Levieuge & Jean-Guillaume Sahuc & Gauthier Vermandel, 2022. "Environmental Subsidies to Mitigate Transition Risk," Swiss Finance Institute Research Paper Series 22-45, Swiss Finance Institute.
    19. Gary Koop & M. Hashem Pesaran & Ron P. Smith, 2013. "On Identification of Bayesian DSGE Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 31(3), pages 300-314, July.
    20. Yuan, Shenguo & Wu, Zhouheng & Liu, Lanfeng, 2022. "The effects of financial openness and financial efficiency on Chinese macroeconomic volatilities," The North American Journal of Economics and Finance, Elsevier, vol. 63(C).
    21. Xiao-Li Gong & Jin-Yan Lu & Xiong Xiong & Wei Zhang, 2025. "Liquidity constraints, real estate regulation, and local government debt risks," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 11(1), pages 1-28, December.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2403.12093. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.