IDEAS home Printed from https://ideas.repec.org/a/eee/dyncon/v143y2022ics0165188922001427.html
   My bibliography  Save this article

Applications of Markov chain approximation methods to optimal control problems in economics

Author

Listed:
  • Phelan, Thomas
  • Eslami, Keyvan

Abstract

In this paper we explore some benefits of using the finite-state Markov chain approximation (MCA) method of Kushner and Dupuis (2001) to solve continuous-time optimal control problems in economics. We first show that the implicit finite-difference scheme of Achdou et al. (2022) amounts to a limiting form of the MCA method for a certain choice of approximating chains and policy function iteration for the resulting system of equations. We then illustrate that relative to the implicit finite-difference approach, using variations of modified policy function iteration to solve income fluctuation problems both with and without discrete choices can lead to an increase in the speed of convergence of more than an order of magnitude. Finally, we provide several consistent chain constructions for stationary portfolio problems with correlated state variables, and illustrate the flexibility of the MCA approach by using it to construct and compare two simple solution methods for a general equilibrium model with financial frictions.

Suggested Citation

  • Phelan, Thomas & Eslami, Keyvan, 2022. "Applications of Markov chain approximation methods to optimal control problems in economics," Journal of Economic Dynamics and Control, Elsevier, vol. 143(C).
    • Handle: RePEc:eee:dyncon:v:143:y:2022:i:c:s0165188922001427
    DOI: 10.1016/j.jedc.2022.104437
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0165188922001427
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jedc.2022.104437?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Markus K. Brunnermeier & Yuliy Sannikov, 2014. "A Macroeconomic Model with a Financial Sector," American Economic Review, American Economic Association, vol. 104(2), pages 379-421, February.
    2. Achdou, Yves & Han, Jiequn & Lasry, Jean Michel & Lions, Pierre Louis & Moll, Ben, 2022. "Income and wealth distribution in macroeconomics: a continuous-time approach," LSE Research Online Documents on Economics 107422, London School of Economics and Political Science, LSE Library.
    3. Daniel Barczyk & Matthias Kredler, 2014. "Altruistically motivated transfers under uncertainty," Quantitative Economics, Econometric Society, vol. 5(3), pages 705-749, November.
    4. Giulio Fella, 2014. "A generalized endogenous grid method for non-smooth and non-concave problems," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 17(2), pages 329-344, April.
    5. Daniel Barczyk & Matthias Kredler, 2014. "A Dynamic Model of Altruistically-Motivated Transfers," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 17(2), pages 303-328, April.
    6. Markus K. Brunnermeier & Yuliy Sannikov, 2016. "Macro, Money and Finance: A Continuous Time Approach," NBER Working Papers 22343, National Bureau of Economic Research, Inc.
    7. Mikhail Golosov & Robert E. Lucas Jr., 2007. "Menu Costs and Phillips Curves," Journal of Political Economy, University of Chicago Press, vol. 115, pages 171-199.
    8. Carroll, Christopher D., 2006. "The method of endogenous gridpoints for solving dynamic stochastic optimization problems," Economics Letters, Elsevier, vol. 91(3), pages 312-320, June.
    9. Yves Achdou & Jiequn Han & Jean-Michel Lasry & Pierre-Louis Lionse & Benjamin Moll, 2022. "Income and Wealth Distribution in Macroeconomics: A Continuous-Time Approach [On the Existence and Uniqueness of Stationary Equilibrium in Bewley Economies with Production]," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 89(1), pages 45-86.
    10. Martin L. Puterman & Moon Chirl Shin, 1978. "Modified Policy Iteration Algorithms for Discounted Markov Decision Problems," Management Science, INFORMS, vol. 24(11), pages 1127-1137, July.
    11. Daniel Barczyk & Matthias Kredler, 2014. "A Dynamic Model of Altruistically-Motivated Transfers," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 17(2), pages 303-328, April.
    12. Martin L. Puterman & Shelby L. Brumelle, 1979. "On the Convergence of Policy Iteration in Stationary Dynamic Programming," Mathematics of Operations Research, INFORMS, vol. 4(1), pages 60-69, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rendahl, Pontus, 2022. "Continuous vs. discrete time: Some computational insights," Journal of Economic Dynamics and Control, Elsevier, vol. 144(C).
    2. Keyvan Eslami & Tom Phelan, 2023. "The Art of Temporal Approximation An Investigation into Numerical Solutions to Discrete and Continuous-Time Problems in Economics," Working Papers 23-10, Federal Reserve Bank of Cleveland.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Keyvan Eslami & Tom Phelan, 2023. "The Art of Temporal Approximation An Investigation into Numerical Solutions to Discrete and Continuous-Time Problems in Economics," Working Papers 23-10, Federal Reserve Bank of Cleveland.
    2. Keyvan Eslami & Tom Phelan, 2021. "Applications of Markov Chain Approximation Methods to Optimal Control Problems in Economics," Working Papers 21-04R, Federal Reserve Bank of Cleveland, revised 17 May 2022.
    3. Daniel Barczyk & Sean Fahle & Matthias Kredler, 2023. "Save, Spend, or Give? A Model of Housing, Family Insurance, and Savings in Old Age," Review of Economic Studies, Oxford University Press, vol. 90(5), pages 2116-2187.
    4. Emmet Hall-Hoffarth, 2023. "Non-linear approximations of DSGE models with neural-networks and hard-constraints," Papers 2310.13436, arXiv.org.
    5. Niko Jaakkola & Florian Wagener & Florian O.O. Wagener, 2023. "Differential Games of Public Investment: Markovian Best Responses in the General Case," CESifo Working Paper Series 10585, CESifo.
    6. Albanesi, Stefania & Gihleb, Rania & Zhang, Ning, 2022. "Boomerang College Kids: Unemployment, Job Mismatch and Coresidence," IZA Discussion Papers 15507, Institute of Labor Economics (IZA).
    7. Ayşe Kabukçuoğlu & Enrique Martínez-García, 2021. "A Generalized Time Iteration Method for Solving Dynamic Optimization Problems with Occasionally Binding Constraints," Computational Economics, Springer;Society for Computational Economics, vol. 58(2), pages 435-460, August.
    8. Ayse Kabukcuoglu & Enrique Martínez-García, 2016. "The Market Resources Method for Solving Dynamic Optimization Problems," Koç University-TUSIAD Economic Research Forum Working Papers 1607, Koc University-TUSIAD Economic Research Forum.
    9. Rendahl, Pontus, 2022. "Continuous vs. discrete time: Some computational insights," Journal of Economic Dynamics and Control, Elsevier, vol. 144(C).
    10. Corina Boar, 2020. "Dynastic Precautionary Savings," NBER Working Papers 26635, National Bureau of Economic Research, Inc.
    11. Daniel Barczyk, 2013. "Deficits, Gifts, and Bequests," 2013 Meeting Papers 25, Society for Economic Dynamics.
    12. Yves Achdou & Jiequn Han & Jean-Michel Lasry & Pierre-Louis Lions & Benjamin Moll, 2017. "Income and Wealth Distribution in Macroeconomics: A Continuous-Time Approach," NBER Working Papers 23732, National Bureau of Economic Research, Inc.
    13. Barczyk, Daniel, 2016. "Ricardian equivalence revisited: Deficits, gifts and bequests," Journal of Economic Dynamics and Control, Elsevier, vol. 63(C), pages 1-24.
    14. Barczyk, Daniel & Kredler, Matthias, 2021. "Blast from the past: The altruism model is richer than you think," Journal of Economic Theory, Elsevier, vol. 198(C).
    15. Dmitriy Sergeyev & Chen Lian & Yuriy Gorodnichenko, 2023. "The Economics of Financial Stress," NBER Working Papers 31285, National Bureau of Economic Research, Inc.
    16. Diego Daruich & Julian Kozlowski, 2020. "Explaining Intergenerational Mobility: The Role of Fertility and Family Transfers," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 36, pages 220-245, April.
    17. Mercedes Esteban-Bravo & Jose M. Vidal-Sanz & Gökhan Yildirim, 2014. "Valuing Customer Portfolios with Endogenous Mass and Direct Marketing Interventions Using a Stochastic Dynamic Programming Decomposition," Marketing Science, INFORMS, vol. 33(5), pages 621-640, September.
    18. Khieu, Hoang & Wälde, Klaus, 2023. "Capital income risk and the dynamics of the wealth distribution," Economic Modelling, Elsevier, vol. 122(C).
    19. Adrien Auclert & Bence Bardóczy & Matthew Rognlie & Ludwig Straub, 2021. "Using the Sequence‐Space Jacobian to Solve and Estimate Heterogeneous‐Agent Models," Econometrica, Econometric Society, vol. 89(5), pages 2375-2408, September.
    20. Koşar, Gizem & Melcangi, Davide & Pilossoph, Laura & Wiczer, David, 2023. "Stimulus through Insurance: The Marginal Propensity to Repay Debt," IZA Discussion Papers 16211, Institute of Labor Economics (IZA).

    More about this item

    Keywords

    Markov chain approximation; Dynamic programming; Numerical methods; Financial frictions;
    All these keywords.

    JEL classification:

    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques
    • E00 - Macroeconomics and Monetary Economics - - General - - - General
    • G11 - Financial Economics - - General Financial Markets - - - Portfolio Choice; Investment Decisions

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:dyncon:v:143:y:2022:i:c:s0165188922001427. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jedc .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.