IDEAS home Printed from https://ideas.repec.org/p/fip/fedcwq/89860.html
   My bibliography  Save this paper

Applications of Markov Chain Approximation Methods to Optimal Control Problems in Economics

Author

Listed:
  • Keyvan Eslami
  • Tom Phelan

Abstract

In this paper we explore some benefits of using the finite-state Markov chain approximation (MCA) method of Kushner and Dupuis (2001) to solve continuous-time optimal control problems in economics. We first show that the implicit finite-difference scheme of Achdou et al. (2022) amounts to a limiting form of the MCA method for a certain choice of approximating chains and policy function iteration for the resulting system of equations. We then illustrate that, relative to the implicit finite-difference approach, using variations of modified policy function iteration to solve income fluctuation problems both with and without discrete choices can lead to an increase in the speed of convergence of more than an order of magnitude. Finally, we provide several consistent chain constructions for stationary portfolio problems with correlated state variables, and illustrate the flexibility of the MCA approach by using it to construct and compare two simple solution methods for a general equilibrium model with financial frictions.

Suggested Citation

  • Keyvan Eslami & Tom Phelan, 2021. "Applications of Markov Chain Approximation Methods to Optimal Control Problems in Economics," Working Papers 21-04R, Federal Reserve Bank of Cleveland, revised 17 May 2022.
    • Handle: RePEc:fip:fedcwq:89860
    DOI: 10.26509/frbc-wp-202104r
    Note: Replication materials may be found at https://github.com/tphelanECON/EslamiPhelan_MCA
    as

    Download full text from publisher

    File URL: https://doi.org/10.26509/frbc-wp-202104r
    File Function: Full Text
    Download Restriction: no
    File URL: https://libkey.io/10.26509/frbc-wp-202104r?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Daniel Barczyk & Matthias Kredler, 2014. "Altruistically motivated transfers under uncertainty," Quantitative Economics, Econometric Society, vol. 5(3), pages 705-749, November.
    2. Daniel Barczyk & Matthias Kredler, 2014. "A Dynamic Model of Altruistically-Motivated Transfers," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 17(2), pages 303-328, April.
    3. Martin L. Puterman & Moon Chirl Shin, 1978. "Modified Policy Iteration Algorithms for Discounted Markov Decision Problems," Management Science, INFORMS, vol. 24(11), pages 1127-1137, July.
    4. Daniel Barczyk & Matthias Kredler, 2014. "A Dynamic Model of Altruistically-Motivated Transfers," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 17(2), pages 303-328, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rendahl, Pontus, 2022. "Continuous vs. discrete time: Some computational insights," Journal of Economic Dynamics and Control, Elsevier, vol. 144(C).
    2. Keyvan Eslami & Tom Phelan, 2023. "The Art of Temporal Approximation An Investigation into Numerical Solutions to Discrete and Continuous-Time Problems in Economics," Working Papers 23-10, Federal Reserve Bank of Cleveland.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Phelan, Thomas & Eslami, Keyvan, 2022. "Applications of Markov chain approximation methods to optimal control problems in economics," Journal of Economic Dynamics and Control, Elsevier, vol. 143(C).
    2. Albanesi, Stefania & Gihleb, Rania & Zhang, Ning, 2022. "Boomerang College Kids: Unemployment, Job Mismatch and Coresidence," IZA Discussion Papers 15507, Institute of Labor Economics (IZA).
    3. Daniel Barczyk & Sean Fahle & Matthias Kredler, 2023. "Save, Spend, or Give? A Model of Housing, Family Insurance, and Savings in Old Age," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(5), pages 2116-2187.
    4. Daniel Barczyk, 2013. "Deficits, Gifts, and Bequests," 2013 Meeting Papers 25, Society for Economic Dynamics.
    5. Barczyk, Daniel, 2016. "Ricardian equivalence revisited: Deficits, gifts and bequests," Journal of Economic Dynamics and Control, Elsevier, vol. 63(C), pages 1-24.
    6. Corina Boar, 2020. "Dynastic Precautionary Savings," NBER Working Papers 26635, National Bureau of Economic Research, Inc.
    7. Barczyk, Daniel & Kredler, Matthias, 2021. "Blast from the past: The altruism model is richer than you think," Journal of Economic Theory, Elsevier, vol. 198(C).
    8. Yves Achdou & Jiequn Han & Jean-Michel Lasry & Pierre-Louis Lionse & Benjamin Moll, 2022. "Income and Wealth Distribution in Macroeconomics: A Continuous-Time Approach [On the Existence and Uniqueness of Stationary Equilibrium in Bewley Economies with Production]," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 89(1), pages 45-86.
    9. Achdou, Yves & Han, Jiequn & Lasry, Jean Michel & Lions, Pierre Louis & Moll, Ben, 2022. "Income and wealth distribution in macroeconomics: a continuous-time approach," LSE Research Online Documents on Economics 107422, London School of Economics and Political Science, LSE Library.
    10. Heer Burkhard & Maußner Alfred, 2011. "Value Function Iteration as a Solution Method for the Ramsey Model," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 231(4), pages 494-515, August.
    11. Greg Kaplan, 2012. "Moving Back Home: Insurance against Labor Market Risk," Journal of Political Economy, University of Chicago Press, vol. 120(3), pages 446-512.
    12. Mauro Gaggero & Giorgio Gnecco & Marcello Sanguineti, 2013. "Dynamic Programming and Value-Function Approximation in Sequential Decision Problems: Error Analysis and Numerical Results," Journal of Optimization Theory and Applications, Springer, vol. 156(2), pages 380-416, February.
    13. Bayer, Christian & Rendall, Alan D. & Wälde, Klaus, 2019. "The invariant distribution of wealth and employment status in a small open economy with precautionary savings," Journal of Mathematical Economics, Elsevier, vol. 85(C), pages 17-37.
    14. Diego Daruich & Julian Kozlowski, 2020. "Explaining Intergenerational Mobility: The Role of Fertility and Family Transfers," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 36, pages 220-245, April.
    15. Gabriel Zayas‐Cabán & Emmett J. Lodree & David L. Kaufman, 2020. "Optimal Control of Parallel Queues for Managing Volunteer Convergence," Production and Operations Management, Production and Operations Management Society, vol. 29(10), pages 2268-2288, October.
    16. Mercedes Esteban-Bravo & Jose M. Vidal-Sanz & Gökhan Yildirim, 2014. "Valuing Customer Portfolios with Endogenous Mass and Direct Marketing Interventions Using a Stochastic Dynamic Programming Decomposition," Marketing Science, INFORMS, vol. 33(5), pages 621-640, September.
    17. Oleksandr Shlakhter & Chi-Guhn Lee, 2013. "Accelerated modified policy iteration algorithms for Markov decision processes," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 78(1), pages 61-76, August.
    18. Corina Boar, 2017. "Dynastic Precautionary Savings," 2017 Meeting Papers 343, Society for Economic Dynamics.
    19. Daniel Barczyk & Matthias Kredler, 2014. "A Dynamic Model of Altruistically-Motivated Transfers," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 17(2), pages 303-328, April.
    20. Marla Ripoll, 2021. "The Patterns of Parental Intervivos Transfers to Adult Children," Working Paper 7144, Department of Economics, University of Pittsburgh.

    More about this item

    Keywords

    Dynamic programming; financial frictions; Numerical methods; Markov chain approximation;
    All these keywords.

    JEL classification:

    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques
    • E00 - Macroeconomics and Monetary Economics - - General - - - General
    • G11 - Financial Economics - - General Financial Markets - - - Portfolio Choice; Investment Decisions

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:fip:fedcwq:89860. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: 4D Library (email available below). General contact details of provider: https://edirc.repec.org/data/frbclus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.