IDEAS home Printed from https://ideas.repec.org/p/cmf/wpaper/wp1999_9904.html
   My bibliography  Save this paper

Swapping the Nested Fixed Point Algorithm: A Class of Estimators for Discrete Markov Decision Models

Author

Listed:
  • Víctor Aguirregabiria
  • Pedro Mira

Abstract

This paper proposes a procedure for the estimation of discrete Markov decision models and studies its statistical and computational properties. Our Nested Pseudo-Likelihood method (NPL) is similar to Rust's Nested Fixed Point algorithm (NFXP), but the order of the two nested algorithms is swapped. First, we prove that NPL produces the Maximum Likelihood Estimator under the same conditions as NFXP. Our procedure requires fewer policy iterations at the expense of more likelihood-climbing iterations. We focus on a class of infinite-horizon, partial likelihood problems for which NPL results in large computational gains. Second, based on this algorithm we define a class of consistent and asymptotically equivalent Sequential Policy Iteration (PI) estimators, which encompasses both Hotz-Miller's CCP estimator and the partial Maximum Likekihood estimator. This presents the researcher with a ''menu'' of sequential estimators reflecting a trade-off between finite-sample precision and computational cost. Using actual and simulated data we compare the relative performance of these estimators. In all our experiments the benefits in terms of precision of using a 2-stage PI estimator instead of 1-stage (i.e., Hotz-Miller) are very significant. More interestingly, the benefits of MLE relative to 2-stage PI are small.

Suggested Citation

  • Víctor Aguirregabiria & Pedro Mira, 1999. "Swapping the Nested Fixed Point Algorithm: A Class of Estimators for Discrete Markov Decision Models," Working Papers wp1999_9904, CEMFI.
  • Handle: RePEc:cmf:wpaper:wp1999_9904
    as

    Download full text from publisher

    File URL: https://www.cemfi.es/ftp/wp/9904.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Rust, John, 1987. "Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher," Econometrica, Econometric Society, vol. 55(5), pages 999-1033, September.
    2. Heckman, James & Singer, Burton, 1984. "A Method for Minimizing the Impact of Distributional Assumptions in Econometric Models for Duration Data," Econometrica, Econometric Society, vol. 52(2), pages 271-320, March.
    3. Hansen, Lars Peter, 1982. "Large Sample Properties of Generalized Method of Moments Estimators," Econometrica, Econometric Society, vol. 50(4), pages 1029-1054, July.
    4. V. Joseph Hotz & Robert A. Miller & Seth Sanders & Jeffrey Smith, 1994. "A Simulation Estimator for Dynamic Models of Discrete Choice," Review of Economic Studies, Oxford University Press, vol. 61(2), pages 265-289.
    5. Gourieroux,Christian & Monfort,Alain, 1995. "Statistics and Econometric Models," Cambridge Books, Cambridge University Press, number 9780521477444, June.
    6. V. Joseph Hotz & Robert A. Miller, 1993. "Conditional Choice Probabilities and the Estimation of Dynamic Models," Review of Economic Studies, Oxford University Press, vol. 60(3), pages 497-529.
    7. Keane, Michael P & Wolpin, Kenneth I, 1994. "The Solution and Estimation of Discrete Choice Dynamic Programming Models by Simulation and Interpolation: Monte Carlo Evidence," The Review of Economics and Statistics, MIT Press, vol. 76(4), pages 648-672, November.
    8. Newey, Whitney K., 1984. "A method of moments interpretation of sequential estimators," Economics Letters, Elsevier, vol. 14(2-3), pages 201-206.
    9. Barnett,William A. & Powell,James & Tauchen,George E. (ed.), 1991. "Nonparametric and Semiparametric Methods in Econometrics and Statistics," Cambridge Books, Cambridge University Press, number 9780521370905, June.
    10. Rust, John, 1996. "Numerical dynamic programming in economics," Handbook of Computational Economics, in: H. M. Amman & D. A. Kendrick & J. Rust (ed.), Handbook of Computational Economics, edition 1, volume 1, chapter 14, pages 619-729, Elsevier.
    11. Keane, Michael P & Wolpin, Kenneth I, 1997. "The Career Decisions of Young Men," Journal of Political Economy, University of Chicago Press, vol. 105(3), pages 473-522, June.
    12. H. M. Amman & D. A. Kendrick & J. Rust (ed.), 1996. "Handbook of Computational Economics," Handbook of Computational Economics, Elsevier, edition 1, volume 1, number 1.
    13. Gourieroux,Christian & Monfort,Alain, 1995. "Statistics and Econometric Models 2 volume set," Cambridge Books, Cambridge University Press, number 9780521478373, July.
    14. John Rust, 1997. "Using Randomization to Break the Curse of Dimensionality," Econometrica, Econometric Society, vol. 65(3), pages 487-516, May.
    15. John Rust, 1997. "A Comparison of Policy Iteration Methods for Solving Continuous-State, Infinite-Horizon Markovian Decision Problems Using Random, Quasi-random, and Deterministic Discretizations," Computational Economics 9704001, University Library of Munich, Germany.
    16. Barnett,William A. & Powell,James & Tauchen,George E. (ed.), 1991. "Nonparametric and Semiparametric Methods in Econometrics and Statistics," Cambridge Books, Cambridge University Press, number 9780521424318, June.
    17. Manski, Charles F., 1993. "Dynamic choice in social settings : Learning from the experiences of others," Journal of Econometrics, Elsevier, vol. 58(1-2), pages 121-136, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Victor Aguirregabiria & Pedro Mira, 2000. "Structural Models Involving Highly Dimensional Fixed Point Problems: An Asymptotically Efficient Two-Stage Estimator," Econometric Society World Congress 2000 Contributed Papers 1702, Econometric Society.
    2. Aguirregabiria, Victor & Mira, Pedro, 2010. "Dynamic discrete choice structural models: A survey," Journal of Econometrics, Elsevier, vol. 156(1), pages 38-67, May.
    3. Keane, Michael P. & Todd, Petra E. & Wolpin, Kenneth I., 2011. "The Structural Estimation of Behavioral Models: Discrete Choice Dynamic Programming Methods and Applications," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 4, pages 331-461, Elsevier.
    4. Victor Aguirregabiria, 2006. "Another Look at the Identification of Dynamic Discrete Decision Processes: With an Application to Retirement Behavior," 2006 Meeting Papers 169, Society for Economic Dynamics.
    5. Andriy Norets, 2009. "Inference in Dynamic Discrete Choice Models With Serially orrelated Unobserved State Variables," Econometrica, Econometric Society, vol. 77(5), pages 1665-1682, September.
    6. Daniel Ackerberg, 2009. "A new use of importance sampling to reduce computational burden in simulation estimation," Quantitative Marketing and Economics (QME), Springer, vol. 7(4), pages 343-376, December.
    7. Elena Mattana & Juanna Joensen, 2016. "Student Aid, Academic Achievement, and Labor Market Behavior," 2016 Meeting Papers 1102, Society for Economic Dynamics.
    8. Heckman, James J. & Navarro, Salvador, 2007. "Dynamic discrete choice and dynamic treatment effects," Journal of Econometrics, Elsevier, vol. 136(2), pages 341-396, February.
    9. Elena Mattana & Juanna Joensen, 2014. "Student Aid, Academic Achievement, and Labor Market Behavior: Grants or Loans?," 2014 Meeting Papers 707, Society for Economic Dynamics.
    10. Sullivan, Paul, 2006. "Interpolating Value Functions in Discrete Choice Dynamic Programming Models," MPRA Paper 864, University Library of Munich, Germany.
    11. Fabio A. Miessi Sanches & Daniel Silva Junior, Sorawoot Srisuma, 2014. "Ordinary Least Squares Estimation for a Dynamic Game," Working Papers, Department of Economics 2014_19, University of São Paulo (FEA-USP), revised 23 Feb 2015.
    12. Aguirregabiria, Victor & Magesan, Arvind, 2013. "Euler Equations for the Estimation of Dynamic Discrete Choice Structural," MPRA Paper 46056, University Library of Munich, Germany.
    13. Yu Zheng & Juan Pantano, 2012. "Using Subjective Expectations Data to Allow for Unobserved Heterogeneity in Hotz-Miller Estimation Strategies," 2012 Meeting Papers 940, Society for Economic Dynamics.
    14. Peter Arcidiacono & Robert A. Miller, 2011. "Conditional Choice Probability Estimation of Dynamic Discrete Choice Models With Unobserved Heterogeneity," Econometrica, Econometric Society, vol. 79(6), pages 1823-1867, November.
    15. Andrew Ching & Susumu Imai & Masakazu Ishihara & Neelam Jain, 2012. "A practitioner’s guide to Bayesian estimation of discrete choice dynamic programming models," Quantitative Marketing and Economics (QME), Springer, vol. 10(2), pages 151-196, June.
    16. Peter Arcidiacono & Holger Sieg & Frank Sloan, 2007. "Living Rationally Under The Volcano? An Empirical Analysis Of Heavy Drinking And Smoking," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 48(1), pages 37-65, February.
    17. Song Lin & Juanjuan Zhang & John R. Hauser, 2015. "Learning from Experience, Simply," Marketing Science, INFORMS, vol. 34(1), pages 1-19, January.
    18. Houser, Daniel, 2003. "Bayesian analysis of a dynamic stochastic model of labor supply and saving," Journal of Econometrics, Elsevier, vol. 113(2), pages 289-335, April.
    19. Juanna Schrøter Joensen, 2010. "Timing and Incentives: Impacts of Student Aid on Academic Achievement," 2010 Meeting Papers 823, Society for Economic Dynamics.
    20. Victor Aguirregabiria & Arvind Magesan, "undated". "Soultion and Estimation of Dynamic Discrete Choice Structural Models Using Euler Equations," Working Papers 2016-32, Department of Economics, University of Calgary, revised 24 May 2016.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cmf:wpaper:wp1999_9904. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/cemfies.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Araceli Requerey (email available below). General contact details of provider: https://edirc.repec.org/data/cemfies.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.