A Comparison of Policy Iteration Methods for Solving Continuous-State, Infinite-Horizon Markovian Decision Problems Using Random, Quasi-random, and Deterministic Discretizations
This paper compares the performance of the Howard (1960) policy iteration algorithm for infinite-horizon continuous-state Markovian decision processes (MDP's) using alternative random, quasi- random, and deterministic discretizations of the state space, or grids. Each grid corresponds to an embedded finite state MDP whose solution is used to approximate the solution to the original continuous-state Markovian decision process. I extend a result of Rust (1997), to show that policy iteration using random grids succeeds in breaking the curse of dimensionality involved in approximating the solution to a class of continuous-state discrete-action MDP's known as discrete decision processes (DDP's). I compare this ``random policy iteration algorithm'' (RPI) with policy iteration algorithms using deterministically chosen grids including uniform grids and quadrature grids both of which are subject to the curse of dimensionality. I also compare the RPI algorithm to deterministic policy iteration algorithms based on quasi-random or `low discrepancy grids' such as the Sobol' and Tezuka sequences.
|Date of creation:||21 Apr 1997|
|Date of revision:|
|Note:||TeX file, Postscript version submitted, 50 pages|
|Contact details of provider:|| Web page: http://econwpa.repec.org|
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Ariel Pakes & Paul McGuire, 1997. "Stochastic Algorithms for Dynamic Models: Markov Perfect Equilibrium, and the 'Curse' of Dimensionality," Cowles Foundation Discussion Papers 1144, Cowles Foundation for Research in Economics, Yale University.
- Keane, Michael P & Wolpin, Kenneth I, 1994.
"The Solution and Estimation of Discrete Choice Dynamic Programming Models by Simulation and Interpolation: Monte Carlo Evidence,"
The Review of Economics and Statistics,
MIT Press, vol. 76(4), pages 648-72, November.
- Michael P. Keane & Kenneth I. Wolpin, 1994. "The solution and estimation of discrete choice dynamic programming models by simulation and interpolation: Monte Carlo evidence," Staff Report 181, Federal Reserve Bank of Minneapolis.
- John Rust, 1997.
"Using Randomization to Break the Curse of Dimensionality,"
Econometric Society, vol. 65(3), pages 487-516, May.
- John Rust & Department of Economics & University of Wisconsin, 1994. "Using Randomization to Break the Curse of Dimensionality," Computational Economics 9403001, EconWPA, revised 04 Jul 1994.
- Rust, J., 1994. "Using Randomization to Break the Curse of Dimensionality," Working papers 9429, Wisconsin Madison - Social Systems.
- Anderson, Evan W. & McGrattan, Ellen R. & Hansen, Lars Peter & Sargent, Thomas J., 1996.
"Mechanics of forming and estimating dynamic linear economies,"
Handbook of Computational Economics,
in: H. M. Amman & D. A. Kendrick & J. Rust (ed.), Handbook of Computational Economics, edition 1, volume 1, chapter 4, pages 171-252
- Lars Peter Hansen & Ellen R. McGrattan & Thomas J. Sargent, 1994. "Mechanics of forming and estimating dynamic linear economies," Staff Report 182, Federal Reserve Bank of Minneapolis.
- Judd, Kenneth L., 1996. "Approximation, perturbation, and projection methods in economic analysis," Handbook of Computational Economics, in: H. M. Amman & D. A. Kendrick & J. Rust (ed.), Handbook of Computational Economics, edition 1, volume 1, chapter 12, pages 509-585 Elsevier.
- Rust, John, 1986. "When Is It Optimal to Kill Off the Market for Used Durable Goods?," Econometrica, Econometric Society, vol. 54(1), pages 65-86, January.
- Rust, John, 1985. "Stationary Equilibrium in a Market for Durable Assets," Econometrica, Econometric Society, vol. 53(4), pages 783-805, July.
- Hans M. Amman & David A. Kendrick, . "Computational Economics," Online economics textbooks, SUNY-Oswego, Department of Economics, number comp1, December.
- Tauchen, George, 1990. "Solving the Stochastic Growth Model by Using Quadrature Methods and Value-Function Iterations," Journal of Business & Economic Statistics, American Statistical Association, vol. 8(1), pages 49-51, January.
- Tauchen, George & Hussey, Robert, 1991. "Quadrature-Based Methods for Obtaining Approximate Solutions to Nonlinear Asset Pricing Models," Econometrica, Econometric Society, vol. 59(2), pages 371-96, March.
When requesting a correction, please mention this item's handle: RePEc:wpa:wuwpco:9704001. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (EconWPA)
If references are entirely missing, you can add them using this form.