Persistently optimal policies in stochastic dynamic programming with generalized discounting
AbstractIn this paper we study a Markov decision process with a non-linear discount function. Our approach is in spirit of the von Neumann-Morgenstern concept and is based on the notion of expectation. First, we define a utility on the space of trajectories of the process in the finite and infinite time horizon and then take their expected values. It turns out that the associated optimization problem leads to a non-stationary dynamic programming and an infinite system of Bellman equations, which result in obtaining persistently optimal policies. Our theory is enriched by examples.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoPaper provided by University Library of Munich, Germany in its series MPRA Paper with number 31755.
Date of creation: 21 Jun 2011
Date of revision:
Stochastic dynamic programming; Persistently optimal policies; Variable discounting; Bellman equation; Resource extraction; Growth theory;
Find related papers by JEL classification:
- D90 - Microeconomics - - Intertemporal Choice and Growth - - - General
- C61 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Optimization Techniques; Programming Models; Dynamic Analysis
This paper has been announced in the following NEP Reports:
- NEP-ALL-2011-07-02 (All new papers)
- NEP-DGE-2011-07-02 (Dynamic General Equilibrium)
- NEP-ORE-2011-07-02 (Operations Research)
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- David M Kreps & Evan L Porteus, 1978.
"Temporal Resolution of Uncertainty and Dynamic Choice Theory,"
Levine's Working Paper Archive
625018000000000009, David K. Levine.
- Kreps, David M & Porteus, Evan L, 1978. "Temporal Resolution of Uncertainty and Dynamic Choice Theory," Econometrica, Econometric Society, vol. 46(1), pages 185-200, January.
- Lucas, Robert Jr. & Stokey, Nancy L., 1984.
"Optimal growth with many consumers,"
Journal of Economic Theory,
Elsevier, vol. 32(1), pages 139-171, February.
- Brock, William A. & Mirman, Leonard J., 1972. "Optimal economic growth and uncertainty: The discounted case," Journal of Economic Theory, Elsevier, vol. 4(3), pages 479-513, June.
- Schäl, Manfred, 1975. "On dynamic programming: Compactness of the space of policies," Stochastic Processes and their Applications, Elsevier, vol. 3(4), pages 345-364, October.
- Boud, John III, 1990. "Recursive utility and the Ramsey problem," Journal of Economic Theory, Elsevier, vol. 50(2), pages 326-345, April.
- Grandmont, Jean-Michel, 1977. "Temporary General Equilibrium Theory," Econometrica, Econometric Society, vol. 45(3), pages 535-72, April.
- Tjalling C. Koopmans, 1959. "Stationary Ordinal Utility and Impatience," Cowles Foundation Discussion Papers 81, Cowles Foundation for Research in Economics, Yale University.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Ekkehart Schlicht).
If references are entirely missing, you can add them using this form.