Convergence Properties of Policy Iteration
AbstractThis paper analyzes the asymptotic convergence properties of policy iteration in a class of stationary, infinite-horizon Markovian decision problems that arise in optimal growth theory. These problems have continuous state and control variables, and must therefore be discretized in order to compute an approximate solution. The discretization converts a potentially infinite dimensional fixed-point problem to a finite dimensional problem defined on a finite grid of points in the state space, and it may thus render inapplicable known convergence results for policy iteration such as those of Puterman and Brumelle (1979). Under certain regularity conditions, we prove that for piecewise linear interpolation, policy iteration converges quadratically, i.e. the sequence of errors en = |Vn - V*| (where Vn is an approximate value function produced from the nth policy iteration step) satisfies en+1 = Le2n for all n. We show how the constant L depends on the grid size of the discretization. Also, under more general conditions we establish that convergence is superlinear. We illustrate the theoretical results with numerical experiments that compare the performance of policy iteration and the method of successive approximations. The quantitative results are consistent with theoretical predictions.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoPaper provided by Department of Economics, W. P. Carey School of Business, Arizona State University in its series Working Papers with number 2133377.
Date of creation:
Date of revision:
This paper has been announced in the following NEP Reports:
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- J. Rust & J. F. Traub & H. Wozniakowski, 2002. "Is There a Curse of Dimensionality for Contraction Fixed Points in the Worst Case?," Econometrica, Econometric Society, vol. 70(1), pages 285-329, January.
- John Rust & Department of Economics & University of Wisconsin, 1994.
"Using Randomization to Break the Curse of Dimensionality,"
9403001, EconWPA, revised 04 Jul 1994.
- John Rust, 1997. "Using Randomization to Break the Curse of Dimensionality," Econometrica, Econometric Society, vol. 65(3), pages 487-516, May.
- Hugo Benitez-Silva & John Rust & Gunter Hitsch & Giorgio Pauletto & George Hall, 2000. "A Comparison Of Discrete And Parametric Methods For Continuous-State Dynamic Programming Problems," Computing in Economics and Finance 2000 24, Society for Computational Economics.
- Santos, Manuel S., 1999. "Numerical solution of dynamic economic models," Handbook of Macroeconomics, in: J. B. Taylor & M. Woodford (ed.), Handbook of Macroeconomics, edition 1, volume 1, chapter 5, pages 311-386 Elsevier.
- Rust, John, 1987. "Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher," Econometrica, Econometric Society, vol. 55(5), pages 999-1033, September.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Steve Salik).
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If references are entirely missing, you can add them using this form.
If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.
Please note that corrections may take a couple of weeks to filter through the various RePEc services.