IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1911.13025.html
   My bibliography  Save this paper

Dynamic Optimal Choice When Rewards are Unbounded Below

Author

Listed:
  • Qingyin Ma
  • John Stachurski

Abstract

We propose a new approach to solving dynamic decision problems with rewards that are unbounded below. The approach involves transforming the Bellman equation in order to convert an unbounded problem into a bounded one. The major advantage is that, when the conditions stated below are satisfied, the transformed problem can be solved by iterating with a contraction mapping. While the method is not universal, we show by example that many common decision problems do satisfy our conditions.

Suggested Citation

  • Qingyin Ma & John Stachurski, 2019. "Dynamic Optimal Choice When Rewards are Unbounded Below," Papers 1911.13025, arXiv.org.
    • Handle: RePEc:arx:papers:1911.13025
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1911.13025
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Rust, John, 1987. "Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher," Econometrica, Econometric Society, vol. 55(5), pages 999-1033, September.
    2. Greg Kaplan & Giovanni L. Violante, 2010. "How Much Consumption Insurance beyond Self-Insurance?," American Economic Journal: Macroeconomics, American Economic Association, vol. 2(4), pages 53-87, October.
    3. Le Van, Cuong & Vailakis, Yiannis, 2005. "Recursive utility and optimal growth with bounded or unbounded returns," Journal of Economic Theory, Elsevier, vol. 123(2), pages 187-209, August.
    4. Jonathan Heathcote & Kjetil Storesletten & Giovanni L. Violante, 2010. "The Macroeconomic Implications of Rising Wage Inequality in the United States," Journal of Political Economy, University of Chicago Press, vol. 118(4), pages 681-722, August.
    5. Andreas Fagereng & Luigi Guiso & Davide Malacrino & Luigi Pistaferri, 2020. "Heterogeneity and Persistence in Returns to Wealth," Econometrica, Econometric Society, vol. 88(1), pages 115-170, January.
    6. Dan Cao, 2018. "Recursive Equilibrium in Krusell and Smith (1998)," Working Papers gueconwpa~18-18-13, Georgetown University, Department of Economics.
    7. Alvarez, Fernando & Stokey, Nancy L., 1998. "Dynamic Programming with Homogeneous Functions," Journal of Economic Theory, Elsevier, vol. 82(1), pages 167-189, September.
    8. J. J. McCall, 1970. "Economics of Information and Job Search," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 84(1), pages 113-126.
    9. Boud, John III, 1990. "Recursive utility and the Ramsey problem," Journal of Economic Theory, Elsevier, vol. 50(2), pages 326-345, April.
    10. Aguiar, Mark & Amador, Manuel, 2019. "A contraction for sovereign debt models," Journal of Economic Theory, Elsevier, vol. 183(C), pages 842-875.
    11. Cristina Arellano, 2008. "Default Risk and Income Fluctuations in Emerging Economies," American Economic Review, American Economic Association, vol. 98(3), pages 690-712, June.
    12. V. Filipe Martins-da-Rocha & Yiannis Vailakis, 2010. "Existence and Uniqueness of a Fixed Point for Local Contractions," Econometrica, Econometric Society, vol. 78(3), pages 1127-1141, May.
    13. Jovanovic, Boyan, 1982. "Selection and the Evolution of Industry," Econometrica, Econometric Society, vol. 50(3), pages 649-670, May.
    14. Li, Huiyu & Stachurski, John, 2014. "Solving the income fluctuation problem with unbounded rewards," Journal of Economic Dynamics and Control, Elsevier, vol. 45(C), pages 353-365.
    15. Benhabib, Jess & Bisin, Alberto & Zhu, Shenghao, 2015. "The wealth distribution in Bewley economies with capital income risk," Journal of Economic Theory, Elsevier, vol. 159(PA), pages 489-515.
    16. Cuong Le Van & Yiannis Vailakis, 2005. "Recursive utility and optimal growth with bounded or unbounded returns," Post-Print halshs-00101201, HAL.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ma, Qingyin & Stachurski, John & Toda, Alexis Akira, 2022. "Unbounded dynamic programming via the Q-transform," Journal of Mathematical Economics, Elsevier, vol. 100(C).
    2. Ma, Qingyin & Stachurski, John, 2019. "Optimal timing of decisions: A general theory based on continuation values," Journal of Economic Dynamics and Control, Elsevier, vol. 101(C), pages 62-81.
    3. Thomas J. Sargent & John Stachurski, 2024. "Dynamic Programming: Finite States," Papers 2401.10473, arXiv.org.
    4. John Stachurski, 2009. "Economic Dynamics: Theory and Computation," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262012774, December.
    5. Philippe Bich & Jean-Pierre Drugeon & Lisa Morhaim, 2015. "On Aggregators and Dynamic Programming," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-01169552, HAL.
    6. John Stachurski & Junnan Zhang, 2019. "Dynamic Programming with State-Dependent Discounting," Papers 1908.08800, arXiv.org, revised Oct 2020.
    7. Philippe Bich & Jean-Pierre Drugeon & Lisa Morhaim, 2018. "On temporal aggregators and dynamic programming," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 66(3), pages 787-817, October.
    8. Philippe Bich & Jean-Pierre Drugeon & Lisa Morhaim, 2015. "On Aggregators and Dynamic Programming," Post-Print halshs-01169552, HAL.
    9. Philippe Bich & Jean-Pierre Drugeon & Lisa Morhaim, 2018. "On Temporal Aggregators and Dynamic Programming," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-01437496, HAL.
    10. Philippe Bich & Jean-Pierre Drugeon & Lisa Morhaim, 2015. "On Aggregators and Dynamic Programming," Documents de travail du Centre d'Economie de la Sorbonne 15053, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
    11. Bloise, Gaetano & Vailakis, Yiannis, 2018. "Convex dynamic programming with (bounded) recursive utility," Journal of Economic Theory, Elsevier, vol. 173(C), pages 118-141.
    12. Robert A. Becker & Juan Pablo Rincón-Zapatero, 2017. "Arbitration and Renegotiation in Trade Agreements," CAEPR Working Papers 2017-007, Center for Applied Economics and Policy Research, Department of Economics, Indiana University Bloomington.
    13. Anna Jaśkiewicz & Janusz Matkowski & Andrzej Nowak, 2014. "On variable discounting in dynamic programming: applications to resource extraction and other economic models," Annals of Operations Research, Springer, vol. 220(1), pages 263-278, September.
    14. Janusz Matkowski & Andrzej Nowak, 2011. "On discounted dynamic programming with unbounded returns," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 46(3), pages 455-474, April.
    15. Becker, Robert A. & Rincón-Zapatero, Juan Pablo, 2021. "Thompson aggregators, Scott continuous Koopmans operators, and Least Fixed Point theory," Mathematical Social Sciences, Elsevier, vol. 112(C), pages 84-97.
    16. Rincón-Zapatero, Juan Pablo, 2022. "Existence and uniqueness of solutions to the Bellman equation in stochastic dynamic programming," UC3M Working papers. Economics 35342, Universidad Carlos III de Madrid. Departamento de Economía.
    17. Guanlong Ren & John Stachurski, 2018. "Dynamic Programming with Recursive Preferences: Optimality and Applications," Papers 1812.05748, arXiv.org, revised Jun 2020.
    18. Robert Becker & Juan Pablo Rincon-Zapatero, 2018. "Recursive Utility and Thompson Aggregators, II: Uniqueness of the Recursive Utility Representation," CAEPR Working Papers 2018-008, Center for Applied Economics and Policy Research, Department of Economics, Indiana University Bloomington.
    19. Cuong Le Van & Lisa Morhaim & Yiannis Vailakis, 2008. "Monotone Concave Operators: An application to the existence and uniqueness of solutions to the Bellman equation," Working Papers hal-00294828, HAL.
    20. Robert Becker & Juan Pablo Rincon-Zapatero, 2018. "Recursive Utility and Thompson Aggregators, I: Constructive Existence Theory for the Koopmans Equation," CAEPR Working Papers 2018-006, Center for Applied Economics and Policy Research, Department of Economics, Indiana University Bloomington.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1911.13025. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.