IDEAS home Printed from https://ideas.repec.org/a/eee/dyncon/v37y2013i1p251-264.html
   My bibliography  Save this article

Fitted value function iteration with probability one contractions

Author

Listed:
  • Pál, Jenő
  • Stachurski, John

Abstract

This paper studies a value function iteration algorithm based on nonexpansive function approximation and Monte Carlo integration that can be applied to almost all stationary dynamic programming problems. The method can be represented using a randomized fitted Bellman operator and a corresponding algorithm that is shown to be globally convergent with probability one. When additional restrictions are imposed, an OP(n−1/2) rate of convergence for Monte Carlo error is obtained.

Suggested Citation

  • Pál, Jenő & Stachurski, John, 2013. "Fitted value function iteration with probability one contractions," Journal of Economic Dynamics and Control, Elsevier, vol. 37(1), pages 251-264.
    • Handle: RePEc:eee:dyncon:v:37:y:2013:i:1:p:251-264
    DOI: 10.1016/j.jedc.2012.08.003
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0165188912001728
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jedc.2012.08.003?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. John Stachurski, 2009. "Economic Dynamics: Theory and Computation," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262012774, December.
    2. John Stachurski, 2008. "Continuous State Dynamic Programming via Nonexpansive Approximation," Computational Economics, Springer;Society for Computational Economics, vol. 31(2), pages 141-160, March.
    3. Kenneth L. Judd, 1998. "Numerical Methods in Economics," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262100711, December.
    4. Pakes, Ariel & McGuire, Paul, 2001. "Stochastic Algorithms, Symmetric Markov Perfect Equilibrium, and the 'Curse' of Dimensionality," Econometrica, Econometric Society, vol. 69(5), pages 1261-1281, September.
    5. Josep Pijoan-Mas, 2006. "Precautionary Savings or Working Longer Hours?," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 9(2), pages 326-352, April.
    6. Kydland, Finn E & Prescott, Edward C, 1982. "Time to Build and Aggregate Fluctuations," Econometrica, Econometric Society, vol. 50(6), pages 1345-1370, November.
    7. S. Rao Aiyagari, 1994. "Uninsured Idiosyncratic Risk and Aggregate Saving," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 109(3), pages 659-684.
    8. Gian Luca Clementi & Hugo A. Hopenhayn, 2006. "A Theory of Financing Constraints and Firm Dynamics," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 121(1), pages 229-265.
    9. Manuel S. Santos & Jesus Vigo-Aguiar, 1998. "Analysis of a Numerical Dynamic Programming Algorithm Applied to Economic Models," Econometrica, Econometric Society, vol. 66(2), pages 409-426, March.
    10. Ábrahám, Árpád & Cárceles-Poveda, Eva, 2010. "Endogenous trading constraints with incomplete asset markets," Journal of Economic Theory, Elsevier, vol. 145(3), pages 974-1004, May.
    11. Aruoba, S. Boragan & Fernandez-Villaverde, Jesus & Rubio-Ramirez, Juan F., 2006. "Comparing solution methods for dynamic equilibrium economies," Journal of Economic Dynamics and Control, Elsevier, vol. 30(12), pages 2477-2508, December.
    12. Huggett, Mark, 1997. "The one-sector growth model with idiosyncratic shocks: Steady states and dynamics," Journal of Monetary Economics, Elsevier, vol. 39(3), pages 385-403, August.
    13. John Rust, 1997. "Using Randomization to Break the Curse of Dimensionality," Econometrica, Econometric Society, vol. 65(3), pages 487-516, May.
    14. Charalambos D. Aliprantis & Kim C. Border, 2006. "Infinite Dimensional Analysis," Springer Books, Springer, edition 0, number 978-3-540-29587-7, November.
    15. Longstaff, Francis A & Schwartz, Eduardo S, 2001. "Valuing American Options by Simulation: A Simple Least-Squares Approach," University of California at Los Angeles, Anderson Graduate School of Management qt43n1k4jb, Anderson Graduate School of Management, UCLA.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Arellano, Cristina & Maliar, Lilia & Maliar, Serguei & Tsyrennikov, Viktor, 2016. "Envelope condition method with an application to default risk models," Journal of Economic Dynamics and Control, Elsevier, vol. 69(C), pages 436-459.
    2. Kristensen, Dennis & Mogensen, Patrick K. & Moon, Jong Myun & Schjerning, Bertel, 2021. "Solving dynamic discrete choice models using smoothing and sieve methods," Journal of Econometrics, Elsevier, vol. 223(2), pages 328-360.
    3. Robert Kirkby Author-Email: robertkirkby@gmail.com|, 2017. "Convergence of Discretized Value Function Iteration," Computational Economics, Springer;Society for Computational Economics, vol. 49(1), pages 117-153, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jenö Pál & John Stachurski, 2011. "Fitted Value Function Iteration With Probability One Contractions," ANU Working Papers in Economics and Econometrics 2011-560, Australian National University, College of Business and Economics, School of Economics.
    2. John Stachurski, 2009. "Economic Dynamics: Theory and Computation," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262012774, December.
    3. Arellano, Cristina & Maliar, Lilia & Maliar, Serguei & Tsyrennikov, Viktor, 2016. "Envelope condition method with an application to default risk models," Journal of Economic Dynamics and Control, Elsevier, vol. 69(C), pages 436-459.
    4. John Stachurski, 2008. "Continuous State Dynamic Programming via Nonexpansive Approximation," Computational Economics, Springer;Society for Computational Economics, vol. 31(2), pages 141-160, March.
    5. Stephanie Becker & Lars Grüne & Willi Semmler, 2007. "Comparing accuracy of second-order approximation and dynamic programming," Computational Economics, Springer;Society for Computational Economics, vol. 30(1), pages 65-91, August.
    6. Aruoba, S. Boragan & Fernandez-Villaverde, Jesus & Rubio-Ramirez, Juan F., 2006. "Comparing solution methods for dynamic equilibrium economies," Journal of Economic Dynamics and Control, Elsevier, vol. 30(12), pages 2477-2508, December.
    7. Yongyang Cai & Kenneth L. Judd, 2023. "A simple but powerful simulated certainty equivalent approximation method for dynamic stochastic problems," Quantitative Economics, Econometric Society, vol. 14(2), pages 651-687, May.
    8. Judd, Kenneth L. & Maliar, Lilia & Maliar, Serguei & Valero, Rafael, 2014. "Smolyak method for solving dynamic economic models: Lagrange interpolation, anisotropic grid and adaptive domain," Journal of Economic Dynamics and Control, Elsevier, vol. 44(C), pages 92-123.
    9. Robert Kirkby Author-Email: robertkirkby@gmail.com|, 2017. "Convergence of Discretized Value Function Iteration," Computational Economics, Springer;Society for Computational Economics, vol. 49(1), pages 117-153, January.
    10. Ayşe Kabukçuoğlu & Enrique Martínez-García, 2021. "A Generalized Time Iteration Method for Solving Dynamic Optimization Problems with Occasionally Binding Constraints," Computational Economics, Springer;Society for Computational Economics, vol. 58(2), pages 435-460, August.
    11. Ayse Kabukcuoglu & Enrique Martínez-García, 2016. "The Market Resources Method for Solving Dynamic Optimization Problems," Koç University-TUSIAD Economic Research Forum Working Papers 1607, Koc University-TUSIAD Economic Research Forum.
    12. Huiyu Li, 2015. "Numerical Policy Error Bounds for $$\eta $$ η -Concave Stochastic Dynamic Programming with Non-interior Solutions," Computational Economics, Springer;Society for Computational Economics, vol. 46(2), pages 171-187, August.
    13. Fernández-Villaverde, J. & Rubio-Ramírez, J.F. & Schorfheide, F., 2016. "Solution and Estimation Methods for DSGE Models," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 527-724, Elsevier.
    14. Cherrier, Beatrice & Duarte, Pedro Garcia & Saïdi, Aurélien, 2023. "Household heterogeneity in macroeconomic models: A historical perspective," European Economic Review, Elsevier, vol. 158(C).
    15. Beatrice Cherrier & Pedro Garcia Duarte & Aurélien Saïdi, 2023. "Household Heterogeneity in Macroeconomic Models: A Historical Perspective," Post-Print hal-04108500, HAL.
    16. Röhrs, Sigrid & Winter, Christoph, 2017. "Reducing government debt in the presence of inequality," Journal of Economic Dynamics and Control, Elsevier, vol. 82(C), pages 1-20.
    17. Franziska Piontek & Matthias Kalkuhl & Elmar Kriegler & Anselm Schultes & Marian Leimbach & Ottmar Edenhofer & Nico Bauer, 2019. "Economic Growth Effects of Alternative Climate Change Impact Channels in Economic Modeling," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 73(4), pages 1357-1385, August.
    18. Geweke, J. & Joel Horowitz & Pesaran, M.H., 2006. "Econometrics: A Bird’s Eye View," Cambridge Working Papers in Economics 0655, Faculty of Economics, University of Cambridge.
    19. Tomi T. Kortela, 2011. "On the costs of disability insurance," 2011 Meeting Papers 445, Society for Economic Dynamics.
    20. Mirman, Leonard J. & Morand, Olivier F. & Reffett, Kevin L., 2008. "A qualitative approach to Markovian equilibrium in infinite horizon economies with capital," Journal of Economic Theory, Elsevier, vol. 139(1), pages 75-98, March.

    More about this item

    Keywords

    Dynamic programming; Value function iteration; Monte Carlo;
    All these keywords.

    JEL classification:

    • C61 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Optimization Techniques; Programming Models; Dynamic Analysis
    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:dyncon:v:37:y:2013:i:1:p:251-264. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jedc .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.