IDEAS home Printed from https://ideas.repec.org/p/zbw/bubdps/512021.html
   My bibliography  Save this paper

Optimal monetary policy using reinforcement learning

Author

Listed:
  • Hinterlang, Natascha
  • Tänzer, Alina

Abstract

This paper introduces a reinforcement learning based approach to compute optimal interest rate reaction functions in terms of fulfilling inflation and output gap targets. The method is generally flexible enough to incorporate restrictions like the zero lower bound, nonlinear economy structures or asymmetric preferences. We use quarterly U.S. data from1987:Q3-2007:Q2 to estimate (nonlinear) model transition equations, train optimal policies and perform counterfactual analyses to evaluate them, assuming that the transition equations remain unchanged. All of our resulting policy rules outperform other common rules as well as the actual federal funds rate. Given a neural network representation of the economy, our optimized nonlinear policy rules reduce the central bank's loss by over43 %. A DSGE model comparison exercise further indicates robustness of the optimized rules.

Suggested Citation

  • Hinterlang, Natascha & Tänzer, Alina, 2021. "Optimal monetary policy using reinforcement learning," Discussion Papers 51/2021, Deutsche Bundesbank.
  • Handle: RePEc:zbw:bubdps:512021
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/248736/1/1784835994.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Laurence Ball & Sandeep Mazumder, 2019. "A Phillips Curve with Anchored Expectations and Short‐Term Unemployment," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 51(1), pages 111-137, February.
    2. Svensson, Lars E. O., 1997. "Inflation forecast targeting: Implementing and monitoring inflation targets," European Economic Review, Elsevier, vol. 41(6), pages 1111-1146, June.
    3. Ioan Carabenciov & Charles Freedman & Mr. Roberto Garcia-Saltos & Mr. Douglas Laxton & Mr. Ondrej Kamenik & Mr. Petar Manchev, 2013. "GPM6: The Global Projection Model with 6 Regions," IMF Working Papers 2013/087, International Monetary Fund.
    4. Andrew Levin & Volker Wieland & John C. Williams, 2003. "The Performance of Forecast-Based Monetary Policy Rules Under Model Uncertainty," American Economic Review, American Economic Association, vol. 93(3), pages 622-645, June.
    5. Olivier Coibion & Yuriy Gorodnichenko, 2015. "Is the Phillips Curve Alive and Well after All? Inflation Expectations and the Missing Disinflation," American Economic Journal: Macroeconomics, American Economic Association, vol. 7(1), pages 197-232, January.
    6. Cúrdia, Vasco & Woodford, Michael, 2016. "Credit Frictions and Optimal Monetary Policy," Journal of Monetary Economics, Elsevier, vol. 84(C), pages 30-65.
    7. Adam, Klaus & Billi, Roberto M., 2006. "Optimal Monetary Policy under Commitment with a Zero Bound on Nominal Interest Rates," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 38(7), pages 1877-1905, October.
    8. Cogan, John F. & Cwik, Tobias & Taylor, John B. & Wieland, Volker, 2010. "New Keynesian versus old Keynesian government spending multipliers," Journal of Economic Dynamics and Control, Elsevier, vol. 34(3), pages 281-295, March.
    9. Bernanke, Ben S. & Gertler, Mark & Gilchrist, Simon, 1999. "The financial accelerator in a quantitative business cycle framework," Handbook of Macroeconomics, in: J. B. Taylor & M. Woodford (ed.), Handbook of Macroeconomics, edition 1, volume 1, chapter 21, pages 1341-1393, Elsevier.
    10. J. Tetlow, Robert & von zur Muehlen, Peter, 2001. "Robust monetary policy with misspecified models: Does model uncertainty always call for attenuated policy?," Journal of Economic Dynamics and Control, Elsevier, vol. 25(6-7), pages 911-949, June.
    11. Marco Del Negro & Marc P. Giannoni & Frank Schorfheide, 2015. "Inflation in the Great Recession and New Keynesian Models," American Economic Journal: Macroeconomics, American Economic Association, vol. 7(1), pages 168-196, January.
    12. Athanasios Orphanides, 2001. "Monetary Policy Rules Based on Real-Time Data," American Economic Review, American Economic Association, vol. 91(4), pages 964-985, September.
    13. Jesús Fernández-Villaverde & Pablo Guerrón-Quintana & Keith Kuester & Juan Rubio-Ramírez, 2015. "Fiscal Volatility Shocks and Economic Activity," American Economic Review, American Economic Association, vol. 105(11), pages 3352-3384, November.
    14. Taylor, John B. & Williams, John C., 2010. "Simple and Robust Rules for Monetary Policy," Handbook of Monetary Economics, in: Benjamin M. Friedman & Michael Woodford (ed.), Handbook of Monetary Economics, edition 1, volume 3, chapter 15, pages 829-859, Elsevier.
    15. Julio J. Rotemberg & Michael Woodford, 1997. "An Optimization-Based Econometric Framework for the Evaluation of Monetary Policy," NBER Chapters, in: NBER Macroeconomics Annual 1997, Volume 12, pages 297-361, National Bureau of Economic Research, Inc.
    16. Schaling, Eric, 2004. "The Nonlinear Phillips Curve and Inflation Forecast Targeting: Symmetric versus Asymmetric Monetary Policy Rules," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 36(3), pages 361-386, June.
    17. Wieland, Volker & Cwik, Tobias & Müller, Gernot J. & Schmidt, Sebastian & Wolters, Maik, 2012. "A new comparative approach to macroeconomic modeling and policy analysis," Journal of Economic Behavior & Organization, Elsevier, vol. 83(3), pages 523-541.
    18. Lars E.O. Svensson, 2020. "Monetary Policy Strategies for the Federal Reserve," International Journal of Central Banking, International Journal of Central Banking, vol. 16(1), pages 133-193, February.
    19. Lawrence J. Christiano & Roberto Motto & Massimo Rostagno, 2014. "Risk Shocks," American Economic Review, American Economic Association, vol. 104(1), pages 27-65, January.
    20. Giorgio E. Primiceri, 2005. "Time Varying Structural Vector Autoregressions and Monetary Policy," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 72(3), pages 821-852.
    21. Wieland, Volker, 2000. "Monetary policy, parameter uncertainty and optimal learning," Journal of Monetary Economics, Elsevier, vol. 46(1), pages 199-228, August.
    22. John B. Taylor & Volker Wieland, 2012. "Surprising Comparative Properties of Monetary Models: Results from a New Model Database," The Review of Economics and Statistics, MIT Press, vol. 94(3), pages 800-816, August.
    23. Stephan Zheng & Alexander Trott & Sunil Srinivasa & Nikhil Naik & Melvin Gruesbeck & David C. Parkes & Richard Socher, 2020. "The AI Economist: Improving Equality and Productivity with AI-Driven Tax Policies," Papers 2004.13332, arXiv.org.
    24. Dolado Juan & Pedrero Ramón María-Dolores & Ruge-Murcia Francisco J., 2004. "Nonlinear Monetary Policy Rules: Some New Evidence for the U.S," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 8(3), pages 1-34, September.
    25. Glenn Rudebusch & Lars E.O. Svensson, 1999. "Policy Rules for Inflation Targeting," NBER Chapters, in: Monetary Policy Rules, pages 203-262, National Bureau of Economic Research, Inc.
    26. Michael Woodford, 2001. "The Taylor Rule and Optimal Monetary Policy," American Economic Review, American Economic Association, vol. 91(2), pages 232-237, May.
    27. Matteo Iacoviello & Stefano Neri, 2010. "Housing Market Spillovers: Evidence from an Estimated DSGE Model," American Economic Journal: Macroeconomics, American Economic Association, vol. 2(2), pages 125-164, April.
    28. Dolado, Juan J. & Maria-Dolores, Ramon & Naveira, Manuel, 2005. "Are monetary-policy reaction functions asymmetric?: The role of nonlinearity in the Phillips curve," European Economic Review, Elsevier, vol. 49(2), pages 485-503, February.
    29. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    30. Orphanides, Athanasios, 2003. "Monetary policy evaluation with noisy information," Journal of Monetary Economics, Elsevier, vol. 50(3), pages 605-631, April.
    31. Gertler, Mark & Karadi, Peter, 2011. "A model of unconventional monetary policy," Journal of Monetary Economics, Elsevier, vol. 58(1), pages 17-34, January.
    32. Taylor, John B., 1993. "Discretion versus policy rules in practice," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 39(1), pages 195-214, December.
    33. Orphanides, Athanasios & Wieland, Volker, 2000. "Inflation zone targeting," European Economic Review, Elsevier, vol. 44(7), pages 1351-1387, June.
    34. Ben S. Bernanke & Julio J. Rotemberg (ed.), 1997. "NBER Macroeconomics Annual 1997," MIT Press Books, The MIT Press, edition 1, volume 1, number 026252242x, December.
    35. Marvin Goodfriend, 2004. "Inflation Targeting in the United States?," NBER Chapters, in: The Inflation-Targeting Debate, pages 311-337, National Bureau of Economic Research, Inc.
    36. Wieland, V. & Afanasyeva, E. & Kuete, M. & Yoo, J., 2016. "New Methods for Macro-Financial Model Comparison and Policy Analysis," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 1241-1319, Elsevier.
    37. Nikolsko-Rzhevskyy, Alex & Papell, David H. & Prodan, Ruxandra, 2014. "Deviations from rules-based policy and their effects," Journal of Economic Dynamics and Control, Elsevier, vol. 49(C), pages 4-17.
    38. Mingli Chen & Andreas Joseph & Michael Kumhof & Xinlei Pan & Xuan Zhou, 2021. "Deep Reinforcement Learning in a Monetary Model," Papers 2104.09368, arXiv.org, revised Jan 2023.
    39. John B Taylor, 2007. "The Explanatory Power of Monetary Policy Rules," Business Economics, Palgrave Macmillan;National Association for Business Economics, vol. 42(4), pages 8-15, October.
    40. Pablo S. Castro & Ajit Desai & Han Du & Rodney Garratt & Francisco Rivadeneyra, 2021. "Estimating Policy Functions in Payments Systems Using Reinforcement Learning," Staff Working Papers 21-7, Bank of Canada.
    41. Lars Peter Hansen & Thomas J Sargent, 2014. "Robust Control and Model Uncertainty," World Scientific Book Chapters, in: UNCERTAINTY WITHIN ECONOMIC MODELS, chapter 5, pages 145-154, World Scientific Publishing Co. Pte. Ltd..
    42. Tambakis Demosthenes N., 2009. "Optimal Monetary Policy with a Convex Phillips Curve," The B.E. Journal of Macroeconomics, De Gruyter, vol. 9(1), pages 1-25, June.
    43. Lucas, Robert Jr, 1976. "Econometric policy evaluation: A critique," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 1(1), pages 19-46, January.
    44. Mark W. Watson, 2014. "Inflation Persistence, the NAIRU, and the Great Recession," American Economic Review, American Economic Association, vol. 104(5), pages 31-36, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ajit Desai, 2023. "Machine Learning for Economics Research: When What and How?," Papers 2304.00086, arXiv.org, revised Apr 2023.
    2. Kshama Dwarakanath & Svitlana Vyetrenko & Peyman Tavallali & Tucker Balch, 2024. "ABIDES-Economist: Agent-Based Simulation of Economic Systems with Learning Agents," Papers 2402.09563, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wieland, Volker & Binder, Michael & Lieberknecht, Philipp & Quintana, Jorge, 2017. "Model Uncertainty in Macroeconomics: On the Implications of Financial Frictions," CEPR Discussion Papers 12013, C.E.P.R. Discussion Papers.
    2. Wieland, Volker & Wolters, Maik, 2013. "Forecasting and Policy Making," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 239-325, Elsevier.
    3. Wieland, V. & Afanasyeva, E. & Kuete, M. & Yoo, J., 2016. "New Methods for Macro-Financial Model Comparison and Policy Analysis," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 1241-1319, Elsevier.
    4. Schmidt, Sebastian & Wieland, Volker, 2013. "The New Keynesian Approach to Dynamic General Equilibrium Modeling: Models, Methods and Macroeconomic Policy Evaluation," Handbook of Computable General Equilibrium Modeling, in: Peter B. Dixon & Dale Jorgenson (ed.), Handbook of Computable General Equilibrium Modeling, edition 1, volume 1, chapter 0, pages 1439-1512, Elsevier.
    5. Wieland, Volker & Cwik, Tobias & Müller, Gernot J. & Schmidt, Sebastian & Wolters, Maik, 2012. "A new comparative approach to macroeconomic modeling and policy analysis," Journal of Economic Behavior & Organization, Elsevier, vol. 83(3), pages 523-541.
    6. Lieberknecht, Philipp, 2018. "Financial Frictions, the Phillips Curve and Monetary Policy," MPRA Paper 89429, University Library of Munich, Germany.
    7. Lindé, J. & Smets, F. & Wouters, R., 2016. "Challenges for Central Banks’ Macro Models," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 2185-2262, Elsevier.
    8. Leitemo, Kai & Lonning, Ingunn, 2006. "Simple Monetary Policymaking without the Output Gap," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 38(6), pages 1619-1640, September.
    9. Lansing, Kevin J. & Trehan, Bharat, 2003. "Forward-looking behavior and optimal discretionary monetary policy," Economics Letters, Elsevier, vol. 81(2), pages 249-256, November.
    10. Taylor, John B. & Williams, John C., 2010. "Simple and Robust Rules for Monetary Policy," Handbook of Monetary Economics, in: Benjamin M. Friedman & Michael Woodford (ed.), Handbook of Monetary Economics, edition 1, volume 3, chapter 15, pages 829-859, Elsevier.
    11. Lars E. O. Svensson, 2003. "What Is Wrong with Taylor Rules? Using Judgment in Monetary Policy through Targeting Rules," Journal of Economic Literature, American Economic Association, vol. 41(2), pages 426-477, June.
    12. Lars E. O. Svensson, 2003. "What Is Wrong with Taylor Rules? Using Judgment in Monetary Policy through Targeting Rules," Journal of Economic Literature, American Economic Association, vol. 41(2), pages 426-477, June.
    13. Matteo Cacciatore & Dmitry Matveev & Rodrigo Sekkel, 2022. "Uncertainty and Monetary Policy Experimentation: Empirical Challenges and Insights from Academic Literature," Discussion Papers 2022-9, Bank of Canada.
    14. Michael Paetz, 2007. "Robust Control and Persistence in the New Keynesian Economy," Quantitative Macroeconomics Working Papers 20711, Hamburg University, Department of Economics.
    15. Lindé, Jesper & Smets, Frank & Wouters, Rafael, 2016. "Challenges for Central Banks´ Macro Models," Working Paper Series 323, Sveriges Riksbank (Central Bank of Sweden).
    16. Aoki, Kosuke, 2003. "On the optimal monetary policy response to noisy indicators," Journal of Monetary Economics, Elsevier, vol. 50(3), pages 501-523, April.
    17. Davide Debortoli & Ricardo Nunes, 2014. "Monetary Regime Switches and Central Bank Preferences," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 46(8), pages 1591-1626, December.
    18. Bodenstein, Martin & Zhao, Junzhu, 2020. "Employment, wages and optimal monetary policy," Journal of Monetary Economics, Elsevier, vol. 112(C), pages 77-96.
    19. Orphanides, Athanasios, 2004. "Monetary Policy Rules, Macroeconomic Stability, and Inflation: A View from the Trenches," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 36(2), pages 151-175, April.
    20. John B. Taylor & Volker Wieland, 2012. "Surprising Comparative Properties of Monetary Models: Results from a New Model Database," The Review of Economics and Statistics, MIT Press, vol. 94(3), pages 800-816, August.

    More about this item

    Keywords

    Optimal Monetary Policy; Reinforcement Learning; Artificial Neural Network; Machine Learning; Reaction Function;
    All these keywords.

    JEL classification:

    • C45 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Neural Networks and Related Topics
    • C61 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Optimization Techniques; Programming Models; Dynamic Analysis
    • E52 - Macroeconomics and Monetary Economics - - Monetary Policy, Central Banking, and the Supply of Money and Credit - - - Monetary Policy
    • E58 - Macroeconomics and Monetary Economics - - Monetary Policy, Central Banking, and the Supply of Money and Credit - - - Central Banks and Their Policies

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:bubdps:512021. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/dbbgvde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.