IDEAS home Printed from https://ideas.repec.org/p/qed/wpaper/1063.html
   My bibliography  Save this paper

Nested Pseudo-likelihood Estimation And Bootstrap-based Inference For Structural Discrete Markov Decision Models

Author

Listed:
  • Hiroyuki Kasahara

    (Department of Economics, University of Western Ontario)

  • Katsumi Shimotsu

    (Department of Economics, Queen's University)

Abstract

This paper analyzes the higher-order properties of nested pseudo-likelihood (NPL) estimators and their practical implementation for parametric discrete Markov decision models in which the probability distribution is defined as a fixed point. We propose a new NPL estimator that can achieve quadratic convergence without fully solving the fixed point problem in every iteration. We then extend the NPL estimators to develop one-step NPL bootstrap procedures for discrete Markov decision models and provide some Monte Carlo evidence based on a machine replacement model of Rust (1987). The proposed one-step bootstrap test statistics and confidence intervals improve upon the first order asymptotics even with a relatively small number of iterations. Improvements are particularly noticeable when analyzing the dynamic impacts of counterfactual policies.

Suggested Citation

  • Hiroyuki Kasahara & Katsumi Shimotsu, 2006. "Nested Pseudo-likelihood Estimation And Bootstrap-based Inference For Structural Discrete Markov Decision Models," Working Paper 1063, Economics Department, Queen's University.
    • Handle: RePEc:qed:wpaper:1063
    as

    Download full text from publisher

    File URL: https://www.econ.queensu.ca/sites/econ.queensu.ca/files/qed_wp_1063.pdf
    File Function: First version 2006
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Rust, John, 1987. "Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher," Econometrica, Econometric Society, vol. 55(5), pages 999-1033, September.
    2. Berkovec, James & Stern, Steven, 1991. "Job Exit Behavior of Older Men," Econometrica, Econometric Society, vol. 59(1), pages 189-210, January.
    3. Andrews, Donald W.K., 2002. "EQUIVALENCE OF THE HIGHER ORDER ASYMPTOTIC EFFICIENCY OF k-STEP AND EXTREMUM STATISTICS," Econometric Theory, Cambridge University Press, vol. 18(5), pages 1040-1085, October.
    4. Victor Aguirregabiria & Pedro Mira, 2002. "Swapping the Nested Fixed Point Algorithm: A Class of Estimators for Discrete Markov Decision Models," Econometrica, Econometric Society, vol. 70(4), pages 1519-1543, July.
    5. V. Joseph Hotz & Robert A. Miller, 1993. "Conditional Choice Probabilities and the Estimation of Dynamic Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 60(3), pages 497-529.
    6. Ichimura, Hidehiko & Lee, Sokbae, 2010. "Characterization of the asymptotic distribution of semiparametric M-estimators," Journal of Econometrics, Elsevier, vol. 159(2), pages 252-266, December.
    7. Tauchen, George, 1986. "Finite state markov-chain approximations to univariate and vector autoregressions," Economics Letters, Elsevier, vol. 20(2), pages 177-181.
    8. Andrews,Donald W. K. & Stock,James H. (ed.), 2005. "Identification and Inference for Econometric Models," Cambridge Books, Cambridge University Press, number 9780521844413.
    9. Donald W. K. Andrews, 2002. "Higher-Order Improvements of a Computationally Attractive "k"-Step Bootstrap for Extremum Estimators," Econometrica, Econometric Society, vol. 70(1), pages 119-162, January.
    10. John Rust & Christopher Phelan, 1997. "How Social Security and Medicare Affect Retirement Behavior in a World of Incomplete Markets," Econometrica, Econometric Society, vol. 65(4), pages 781-832, July.
    11. Robinson, Peter M, 1988. "The Stochastic Difference between Econometric Statistics," Econometrica, Econometric Society, vol. 56(3), pages 531-548, May.
    12. Miller, Robert A, 1984. "Job Matching and Occupational Choice," Journal of Political Economy, University of Chicago Press, vol. 92(6), pages 1086-1120, December.
    13. Pakes, Ariel S, 1986. "Patents as Options: Some Estimates of the Value of Holding European Patent Stocks," Econometrica, Econometric Society, vol. 54(4), pages 755-784, July.
    14. Davidson, Russell & MacKinnon, James G., 1999. "The Size Distortion Of Bootstrap Tests," Econometric Theory, Cambridge University Press, vol. 15(3), pages 361-376, June.
    15. John Haltiwanger & Russell Cooper & Laura Power, 1999. "Machine Replacement and the Business Cycle: Lumps and Bumps," American Economic Review, American Economic Association, vol. 89(4), pages 921-946, September.
    16. Unknown, 1986. "Letters," Choices: The Magazine of Food, Farm, and Resource Issues, Agricultural and Applied Economics Association, vol. 1(4), pages 1-9.
    17. Davidson, Russell & MacKinnon, James G, 1999. "Bootstrap Testing in Nonlinear Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 40(2), pages 487-508, May.
    18. Rust, John, 1996. "Numerical dynamic programming in economics," Handbook of Computational Economics, in: H. M. Amman & D. A. Kendrick & J. Rust (ed.), Handbook of Computational Economics, edition 1, volume 1, chapter 14, pages 619-729, Elsevier.
    19. Keane, Michael P & Wolpin, Kenneth I, 1997. "The Career Decisions of Young Men," Journal of Political Economy, University of Chicago Press, vol. 105(3), pages 473-522, June.
    20. H. M. Amman & D. A. Kendrick & J. Rust (ed.), 1996. "Handbook of Computational Economics," Handbook of Computational Economics, Elsevier, edition 1, volume 1, number 1.
    21. Donna B. Gilleskie, 1998. "A Dynamic Stochastic Model of Medical Care Use and Work Absence," Econometrica, Econometric Society, vol. 66(1), pages 1-46, January.
    22. Susumu Imai & Michael P. Keane, 2004. "Intertemporal Labor Supply and Human Capital Accumulation," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 45(2), pages 601-641, May.
    23. Donald W.K. Andrews, 2001. "Higher-order Improvements of the Parametric Bootstrap for Markov Processes," Cowles Foundation Discussion Papers 1334, Cowles Foundation for Research in Economics, Yale University.
    24. Hans M. Amman & David A. Kendrick, . "Computational Economics," Online economics textbooks, SUNY-Oswego, Department of Economics, number comp1.
    25. Zvi Eckstein & Kenneth I. Wolpin, 1999. "Why Youths Drop Out of High School: The Impact of Preferences, Opportunities, and Abilities," Econometrica, Econometric Society, vol. 67(6), pages 1295-1340, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Aguirregabiria, Victor & Mira, Pedro, 2010. "Dynamic discrete choice structural models: A survey," Journal of Econometrics, Elsevier, vol. 156(1), pages 38-67, May.
    2. Hiroyuki Kasahara & Katsumi Shimotsu, 2012. "Sequential Estimation of Structural Models With a Fixed Point Constraint," Econometrica, Econometric Society, vol. 80(5), pages 2303-2319, September.
    3. Hiroyuki Kasahara & Katsumi Shimotsu, 2006. "Nonparametric Identification And Estimation Of Finite Mixture Models Of Dynamic Discrete Choices," Working Paper 1092, Economics Department, Queen's University.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kasahara, Hiroyuki & Shimotsu, Katsumi, 2008. "Pseudo-likelihood estimation and bootstrap inference for structural discrete Markov decision models," Journal of Econometrics, Elsevier, vol. 146(1), pages 92-106, September.
    2. Aguirregabiria, Victor & Mira, Pedro, 2010. "Dynamic discrete choice structural models: A survey," Journal of Econometrics, Elsevier, vol. 156(1), pages 38-67, May.
    3. Victor Aguirregabiria, 2006. "Another Look at the Identification of Dynamic Discrete Decision Processes: With an Application to Retirement Behavior," 2006 Meeting Papers 169, Society for Economic Dynamics.
    4. Sebastian Galiani & Juan Pantano, 2021. "Structural Models: Inception and Frontier," NBER Working Papers 28698, National Bureau of Economic Research, Inc.
    5. Peter Arcidiacono & Holger Sieg & Frank Sloan, 2007. "Living Rationally Under The Volcano? An Empirical Analysis Of Heavy Drinking And Smoking," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 48(1), pages 37-65, February.
    6. Kalouptsidi, Myrto & Scott, Paul & Souza-Rodrigues, Edouardo, 2015. "Identification of Counterfactuals and Payoffs in Dynamic Discrete Choice with an Application to Land Use," TSE Working Papers 15-596, Toulouse School of Economics (TSE).
    7. Hiroyuki Kasahara & Katsumi Shimotsu, 2018. "Estimation of Discrete Choice Dynamic Programming Models," The Japanese Economic Review, Japanese Economic Association, vol. 69(1), pages 28-58, March.
    8. Le-Yu Chen, 2009. "Identification of structural dynamic discrete choice models," CeMMAP working papers CWP08/09, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    9. Gamba, Andrea & Tesser, Matteo, 2009. "Structural estimation of real options models," Journal of Economic Dynamics and Control, Elsevier, vol. 33(4), pages 798-816, April.
    10. Myrto Kalouptsidi & Paul T. Scott & Eduardo Souza-Rodrigues, 2015. "Identification of Counterfactuals in Dynamic Discrete Choice Models," NBER Working Papers 21527, National Bureau of Economic Research, Inc.
    11. Peter Arcidiacono, Holger Sieg, Frank Sloan, 2001. "Living Rationally Under the Volcano? Heavy Drinking and Smoking Among the Elderly," Computing in Economics and Finance 2001 207, Society for Computational Economics.
    12. Sullivan, Paul, 2006. "Interpolating Value Functions in Discrete Choice Dynamic Programming Models," MPRA Paper 864, University Library of Munich, Germany.
    13. Victor Aguirregabiria, 2005. "Another Look at the Identification of Dynamic Discrete Decision Processes," Econometrics 0504006, University Library of Munich, Germany.
    14. Keane, Michael P. & Todd, Petra E. & Wolpin, Kenneth I., 2011. "The Structural Estimation of Behavioral Models: Discrete Choice Dynamic Programming Methods and Applications," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 4, pages 331-461, Elsevier.
    15. Peter Arcidiacono & Robert A. Miller, 2011. "Conditional Choice Probability Estimation of Dynamic Discrete Choice Models With Unobserved Heterogeneity," Econometrica, Econometric Society, vol. 79(6), pages 1823-1867, November.
    16. Maria Casanova-Rivas, 2008. "Dynamic Complementarities: A Computational and Empirical Analysis of Couples' Retirement Decisions," 2008 Meeting Papers 1073, Society for Economic Dynamics.
    17. Andriy Norets, 2009. "Inference in Dynamic Discrete Choice Models With Serially orrelated Unobserved State Variables," Econometrica, Econometric Society, vol. 77(5), pages 1665-1682, September.
    18. Victor Aguirregabiria & Arvind Magesan, "undated". "Soultion and Estimation of Dynamic Discrete Choice Structural Models Using Euler Equations," Working Papers 2016-32, Department of Economics, University of Calgary, revised 24 May 2016.
    19. Paul Sullivan, 2010. "A Dynamic Analysis Of Educational Attainment, Occupational Choices, And Job Search," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 51(1), pages 289-317, February.
    20. Heckman, James J. & Navarro, Salvador, 2007. "Dynamic discrete choice and dynamic treatment effects," Journal of Econometrics, Elsevier, vol. 136(2), pages 341-396, February.

    More about this item

    Keywords

    Edgeworth expansion; k-step bootstrap; maximum pseudo-likelihood estimators; nested fixed point algorithm; Newton-Raphson method; policy iteration;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C13 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Estimation: General
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • C44 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Operations Research; Statistical Decision Theory
    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:qed:wpaper:1063. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Mark Babcock (email available below). General contact details of provider: https://edirc.repec.org/data/qedquca.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.