IDEAS home Printed from
MyIDEAS: Log in (now much improved!) to save this article

Pseudo-likelihood estimation and bootstrap inference for structural discrete Markov decision models

  • Kasahara, Hiroyuki
  • Shimotsu, Katsumi

This paper analyzes the higher-order properties of the estimators based on the nested pseudo-likelihood (NPL) algorithm and the practical implementation of such estimators for parametric discrete Markov decision models. We derive the rate at which the NPL algorithm converges to the MLE and provide a theoretical explanation for the simulation results in Aguirregabiria and Mira [Aguirregabiria, V., Mira, P., 2002. Swapping the nested fixed point algorithm: A class of estimators for discrete Markov decision models. Econometrica 70, 1519-1543], in which iterating the NPL algorithm improves the accuracy of the estimator. We then propose a new NPL algorithm that can achieve quadratic convergence without fully solving the fixed point problem in every iteration and apply our estimation procedure to a finite mixture model. We also develop one-step NPL bootstrap procedures for discrete Markov decision models. The Monte Carlo simulation evidence based on a machine replacement model of Rust [Rust, J., 1987. Optimal replacement of GMC bus engines: An empirical model of Harold Zurcher. Econometrica 55, 999-1033] shows that the proposed one-step bootstrap test statistics and confidence intervals improve upon the first order asymptotics even with a relatively small number of iterations.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL:
Download Restriction: Full text for ScienceDirect subscribers only

As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

Article provided by Elsevier in its journal Journal of Econometrics.

Volume (Year): 146 (2008)
Issue (Month): 1 (September)
Pages: 92-106

in new window

Handle: RePEc:eee:econom:v:146:y:2008:i:1:p:92-106
Contact details of provider: Web page:

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Russell Cooper & John Haltiwanger & Laura Power, 1995. "Machine Replacement and the Business Cycle: Lumps and Bumps," Papers 0062, Boston University - Industry Studies Programme.
  2. Pakes, Ariel S, 1986. "Patents as Options: Some Estimates of the Value of Holding European Patent Stocks," Econometrica, Econometric Society, vol. 54(4), pages 755-84, July.
  3. Hiroyuki Kasahara & Beverly Lapham, 2008. "Productivity and the Decision to Import and Export: Theory and Evidence," CESifo Working Paper Series 2240, CESifo Group Munich.
  4. Donald W. K. Andrews, 2002. "Higher-Order Improvements of a Computationally Attractive "k"-Step Bootstrap for Extremum Estimators," Econometrica, Econometric Society, vol. 70(1), pages 119-162, January.
  5. Bajari, Patrick & Benkard, C. Lanier & Levin, Jonathan, 2007. "Estimating Dynamic Models of Imperfect Competition," Research Papers 1852r1, Stanford University, Graduate School of Business.
  6. Susumu Imai & Neelam Jain & Andrew Ching, 2006. "Bayesian Estimation of Dynamic Discrete Choice Models," Working Papers 1118, Queen's University, Department of Economics.
  7. V. Joseph Hotz & Robert A. Miller, 1993. "Conditional Choice Probabilities and the Estimation of Dynamic Models," Review of Economic Studies, Oxford University Press, vol. 60(3), pages 497-529.
  8. Rust, John, 1987. "Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher," Econometrica, Econometric Society, vol. 55(5), pages 999-1033, September.
  9. Miller, Robert A. & Sanders, Seth G., 1997. "Human capital development and welfare participation," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 46(1), pages 1-43, June.
  10. Victor Aguirregabiria & Pedro Mira, 2004. "Sequential Estimation Of Dynamic Discrete Games," Working Papers wp2004_0413, CEMFI.
  11. Ariel Pakes & Michael Ostrovsky & Steve Berry, 2004. "Simple Estimators for the Parameters of Discrete Dynamic Games (with Entry/Exit Samples)," NBER Working Papers 10506, National Bureau of Economic Research, Inc.
  12. Donald W.K. Andrews, 2000. "Equivalence of the Higher-order Asymptotic Efficiency of k-step and Extremum Statistics," Cowles Foundation Discussion Papers 1269, Cowles Foundation for Research in Economics, Yale University.
  13. Kennet, D Mark, 1994. "A Structural Model of Aircraft Engine Maintenance," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 9(4), pages 351-68, Oct.-Dec..
  14. Gotz, Glenn A. & McCall, John J., 1980. "Estimation in sequential decisionmaking models : A methodological note," Economics Letters, Elsevier, vol. 6(2), pages 131-136.
  15. Hidehiko Ichimura & Sokbae Lee, 2010. "Characterization of the asymptotic distribution of semiparametric M-estimators," Post-Print hal-00741628, HAL.
  16. John Rust & Christopher Phelan, 1994. "How Social Security and Medicare Affect Retirement Behavior in a World of Incomplete Markets," Public Economics 9406005, EconWPA, revised 06 Jul 1994.
  17. John Kennan & James R. Walker, 2011. "The Effect of Expected Income on Individual Migration Decisions," Econometrica, Econometric Society, vol. 79(1), pages 211-251, 01.
  18. Hiroyuki Kasahara & Katsumi Shimotsu, 2006. "Nonparametric Identification and Estimation of Finite Mixture Models of Dynamic Discrete Choices," Working Papers 1092, Queen's University, Department of Economics.
  19. Paola Rota, 2004. "Estimating Labor Demand with Fixed Costs," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 45(1), pages 25-48, 02.
  20. Arcidiacono, Peter, 2004. "Ability sorting and the returns to college major," Journal of Econometrics, Elsevier, vol. 121(1-2), pages 343-375.
  21. Victor Aguirregabiria & Pedro Mira, 2002. "Swapping the Nested Fixed Point Algorithm: A Class of Estimators for Discrete Markov Decision Models," Econometrica, Econometric Society, vol. 70(4), pages 1519-1543, July.
  22. Keane, Michael P & Wolpin, Kenneth I, 1997. "The Career Decisions of Young Men," Journal of Political Economy, University of Chicago Press, vol. 105(3), pages 473-522, June.
  23. Susumu Imai & Michael P. Keane, 2004. "Intertemporal Labor Supply and Human Capital Accumulation," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 45(2), pages 601-641, 05.
  24. Arjan Heyma, 2004. "A structural dynamic analysis of retirement behaviour in the Netherlands," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 19(6), pages 739-759.
  25. Sumru Altuğ & Robert A. Miller, 1998. "The Effect of Work Experience on Female Wages and Labour Supply," Review of Economic Studies, Oxford University Press, vol. 65(1), pages 45-85.
  26. Davidson, R. & Mackinnon, J.G., 1997. "Bootstrap Testing in Nonlinear Models," G.R.E.Q.A.M. 97a39, Universite Aix-Marseille III.
  27. Margaret E. Slade & G.R.E.Q.A.M., 1998. "Optimal Pricing with Costly Adjustment: Evidence from Retail-Grocery Prices," Review of Economic Studies, Oxford University Press, vol. 65(1), pages 87-107.
  28. Peter Arcidiacono, 2005. "Affirmative Action in Higher Education: How Do Admission and Financial Aid Rules Affect Future Earnings?," Econometrica, Econometric Society, vol. 73(5), pages 1477-1524, 09.
  29. Patrick Bajari & Han Hong, 2006. "Semiparametric Estimation of a Dynamic Game of Incomplete Information," NBER Technical Working Papers 0320, National Bureau of Economic Research, Inc.
  30. Rothwell, Geoffrey & Rust, John, 1997. "On the Optimal Lifetime of Nuclear Power Plants," Journal of Business & Economic Statistics, American Statistical Association, vol. 15(2), pages 195-208, April.
  31. Victor Aguirregabiria, 1999. "The Dynamics of Markups and Inventories in Retailing Firms," Review of Economic Studies, Oxford University Press, vol. 66(2), pages 275-308.
  32. Davidson, Russell & MacKinnon, James G., 1999. "The Size Distortion Of Bootstrap Tests," Econometric Theory, Cambridge University Press, vol. 15(03), pages 361-376, June.
  33. Donna B. Gilleskie, 1998. "A Dynamic Stochastic Model of Medical Care Use and Work Absence," Econometrica, Econometric Society, vol. 66(1), pages 1-46, January.
  34. Martin Pesendorfer & Philipp Schmidt-Dengler, 2008. "Asymptotic Least Squares Estimators for Dynamic Games -super-1," Review of Economic Studies, Oxford University Press, vol. 75(3), pages 901-928.
  35. Zvi Eckstein & Kenneth I. Wolpin, 1999. "Why Youths Drop Out of High School: The Impact of Preferences, Opportunities, and Abilities," Econometrica, Econometric Society, vol. 67(6), pages 1295-1340, November.
  36. Gonul, Fusun F., 1998. "Estimating price expectations in the OTC medicine market: An application of dynamic stochastic discrete choice models to scanner panel data," Journal of Econometrics, Elsevier, vol. 89(1-2), pages 41-56, November.
  37. Miller, Robert A, 1984. "Job Matching and Occupational Choice," Journal of Political Economy, University of Chicago Press, vol. 92(6), pages 1086-120, December.
  38. Wolpin, Kenneth I, 1984. "An Estimable Dynamic Stochastic Model of Fertility and Child Mortality," Journal of Political Economy, University of Chicago Press, vol. 92(5), pages 852-74, October.
  39. Robinson, Peter M, 1988. "The Stochastic Difference between Econometric Statistics," Econometrica, Econometric Society, vol. 56(3), pages 531-48, May.
  40. Namkee Ahn, 1995. "Measuring the Value of Children by Sex and Age Using a Dynamic Programming Model," Review of Economic Studies, Oxford University Press, vol. 62(3), pages 361-379.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:146:y:2008:i:1:p:92-106. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Shamier, Wendy)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.