IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/31755.html

Persistently optimal policies in stochastic dynamic programming with generalized discounting

Author

Listed:
  • Jaśkiewicz, Anna
  • Matkowski, Janusz
  • Nowak, Andrzej S.

Abstract

In this paper we study a Markov decision process with a non-linear discount function. Our approach is in spirit of the von Neumann-Morgenstern concept and is based on the notion of expectation. First, we define a utility on the space of trajectories of the process in the finite and infinite time horizon and then take their expected values. It turns out that the associated optimization problem leads to a non-stationary dynamic programming and an infinite system of Bellman equations, which result in obtaining persistently optimal policies. Our theory is enriched by examples.

Suggested Citation

  • Jaśkiewicz, Anna & Matkowski, Janusz & Nowak, Andrzej S., 2011. "Persistently optimal policies in stochastic dynamic programming with generalized discounting," MPRA Paper 31755, University Library of Munich, Germany.
    • Handle: RePEc:pra:mprapa:31755
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/31755/1/MPRA_paper_31755.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Grandmont, Jean-Michel, 1993. "Temporary general equilibrium theory," Handbook of Mathematical Economics, in: K. J. Arrow & M.D. Intriligator (ed.), Handbook of Mathematical Economics, edition 4, volume 2, chapter 19, pages 879-922, Elsevier.
    2. Tjalling C. Koopmans, 1959. "Stationary Ordinal Utility and Impatience," Cowles Foundation Discussion Papers 81, Cowles Foundation for Research in Economics, Yale University.
    3. Boud, John III, 1990. "Recursive utility and the Ramsey problem," Journal of Economic Theory, Elsevier, vol. 50(2), pages 326-345, April.
    4. Lucas, Robert Jr. & Stokey, Nancy L., 1984. "Optimal growth with many consumers," Journal of Economic Theory, Elsevier, vol. 32(1), pages 139-171, February.
    5. William A. Brock & Leonard J. Mirman, 2001. "Optimal Economic Growth And Uncertainty: The Discounted Case," Chapters, in: W. D. Dechert (ed.), Growth Theory, Nonlinear Dynamics and Economic Modelling, chapter 1, pages 3-37, Edward Elgar Publishing.
    6. Schäl, Manfred, 1975. "On dynamic programming: Compactness of the space of policies," Stochastic Processes and their Applications, Elsevier, vol. 3(4), pages 345-364, October.
    7. Paul A. Samuelson, 1937. "A Note on Measurement of Utility," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 4(2), pages 155-161.
    8. Kreps, David M & Porteus, Evan L, 1978. "Temporal Resolution of Uncertainty and Dynamic Choice Theory," Econometrica, Econometric Society, vol. 46(1), pages 185-200, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Anna Jaśkiewicz & Janusz Matkowski & Andrzej Nowak, 2014. "On variable discounting in dynamic programming: applications to resource extraction and other economic models," Annals of Operations Research, Springer, vol. 220(1), pages 263-278, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. A. Jaśkiewicz & J. Matkowski & A. S. Nowak, 2013. "Persistently Optimal Policies in Stochastic Dynamic Programming with Generalized Discounting," Mathematics of Operations Research, INFORMS, vol. 38(1), pages 108-121, February.
    2. Anna Jaśkiewicz & Janusz Matkowski & Andrzej Nowak, 2014. "On variable discounting in dynamic programming: applications to resource extraction and other economic models," Annals of Operations Research, Springer, vol. 220(1), pages 263-278, September.
    3. Thomas J. Sargent & John Stachurski, 2024. "Dynamic Programming: Finite States," Papers 2401.10473, arXiv.org.
    4. Bloise, Gaetano & Vailakis, Yiannis, 2018. "Convex dynamic programming with (bounded) recursive utility," Journal of Economic Theory, Elsevier, vol. 173(C), pages 118-141.
    5. Bloise, G. & Van, C. Le & Vailakis, Y., 2024. "An approximation approach to dynamic programming with unbounded returns," Journal of Mathematical Economics, Elsevier, vol. 111(C).
    6. Guanlong Ren & John Stachurski, 2018. "Dynamic Programming with Recursive Preferences: Optimality and Applications," Papers 1812.05748, arXiv.org, revised Jun 2020.
    7. Farmer, Roger E.A. & Lahiri, Amartya, 2005. "Recursive preferences and balanced growth," Journal of Economic Theory, Elsevier, vol. 125(1), pages 61-77, November.
    8. Jorge Durán, 2003. "Discounting long run average growth in stochastic dynamic programs," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 22(2), pages 395-413, September.
    9. Harashima, Taiji, 2009. "Depression as a Nash Equilibrium Consisting of Strategies of Choosing a Pareto Inefficient Transition Path," MPRA Paper 18953, University Library of Munich, Germany.
    10. Joshi, Sumit, 1995. "Recursive utility and optimal growth under uncertainty," Journal of Mathematical Economics, Elsevier, vol. 24(6), pages 601-617.
    11. Ozaki, Hiroyuki & Streufert, Peter A., 1996. "Dynamic programming for non-additive stochastic objectives," Journal of Mathematical Economics, Elsevier, vol. 25(4), pages 391-442.
    12. Harashima, Taiji, 2011. "A Mechanism of Cyclical Volatility in the Vacancy-Unemployment Ratio: What Is the Source of Rigidity?," MPRA Paper 32476, University Library of Munich, Germany.
    13. Jaroslav Borovička & John Stachurski, 2020. "Necessary and Sufficient Conditions for Existence and Uniqueness of Recursive Utilities," Journal of Finance, American Finance Association, vol. 75(3), pages 1457-1493, June.
    14. Marinacci, Massimo & Montrucchio, Luigi, 2010. "Unique solutions for stochastic recursive utilities," Journal of Economic Theory, Elsevier, vol. 145(5), pages 1776-1804, September.
    15. Stéphane Zuber, 2011. "The aggregation of preferences: can we ignore the past?," Theory and Decision, Springer, vol. 70(3), pages 367-384, March.
    16. David K. Backus & Bryan R. Routledge & Stanley E. Zin, 2005. "Exotic Preferences for Macroeconomists," NBER Chapters, in: NBER Macroeconomics Annual 2004, Volume 19, pages 319-414, National Bureau of Economic Research, Inc.
    17. Juan Rincón-Zapatero & Carlos Rodríguez-Palmero, 2007. "Recursive utility with unbounded aggregators," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 33(2), pages 381-391, November.
    18. Charalambos Aliprantis & Kim Border & Owen Burkinshaw, 1996. "Market economies with many commodities," Decisions in Economics and Finance, Springer;Associazione per la Matematica, vol. 19(1), pages 113-185, March.
    19. Li, Hanwu & Riedel, Frank & Yang, Shuzhen, 2024. "Optimal consumption for recursive preferences with local substitution — the case of certainty," Journal of Mathematical Economics, Elsevier, vol. 110(C).
    20. John Stachurski, 2009. "Economic Dynamics: Theory and Computation," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262012774, December.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • D90 - Microeconomics - - Micro-Based Behavioral Economics - - - General
    • C61 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Optimization Techniques; Programming Models; Dynamic Analysis

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:31755. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.