IDEAS home Printed from
MyIDEAS: Login to save this paper or follow this series

Sensitivity of Propensity Score Methods to the Specifications

  • Zhao, Zhong


    (Renmin University of China)

Propensity score matching estimators have two advantages. One is that they overcome the curse of dimensionality of covariate matching, and the other is that they are nonparametric. However, the propensity score is usually unknown and needs to be estimated. If we estimate it nonparametrically, we are incurring the curse-of-dimensionality problem we are trying to avoid. If we estimate it parametrically, how sensitive the estimated treatment effects are to the specifications of the propensity score becomes an important question. In this paper, we study this issue. First, we use a Monte Carlo experimental method to investigate the sensitivity issue under the unconfoundedness assumption. We find that the estimates are not sensitive to the specifications. Next, we provide some theoretical justifications, using the insight from Rosenbaum and Rubin (1983) that any score finer than the propensity score is a balancing score. Then, we reconcile our finding with the finding in Smith and Todd (2005) that, if the unconfoundedness assumption fails, the matching results can be sensitive. However, failure of the unconfoundedness assumption will not necessarily result in sensitive estimates. Matching estimators can be speciously robust in the sense that the treatment effects are consistently overestimated or underestimated. Sensitivity checks applied in empirical studies are helpful in eliminating sensitive cases, but in general, it cannot help to solve the fundamental problem that the matching assumptions are inherently untestable. Last, our results suggest that including irrelevant variables in the propensity score will not bias the results, but overspecifying it (e.g., adding unnecessary nonlinear terms) probably will.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL:
Download Restriction: no

Paper provided by Institute for the Study of Labor (IZA) in its series IZA Discussion Papers with number 1873.

in new window

Length: 37 pages
Date of creation: Dec 2005
Date of revision:
Handle: RePEc:iza:izadps:dp1873
Contact details of provider: Postal: IZA, P.O. Box 7240, D-53072 Bonn, Germany
Phone: +49 228 3894 223
Fax: +49 228 3894 180
Web page:

Order Information: Postal: IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Mueser, Peter R. & Troske, Kenneth & Gorislavsky, Alexey, 2003. "Using State Administrative Data to Measure Program Performance," IZA Discussion Papers 786, Institute for the Study of Labor (IZA).
  2. Jinyong Hahn, 1998. "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica, Econometric Society, vol. 66(2), pages 315-332, March.
  3. James J. Heckman & Edward Vytlacil, 2005. "Structural Equations, Treatment Effects and Econometric Policy Evaluation," NBER Technical Working Papers 0306, National Bureau of Economic Research, Inc.
  4. Guido W. Imbens, 1999. "The Role of the Propensity Score in Estimating Dose-Response Functions," NBER Technical Working Papers 0237, National Bureau of Economic Research, Inc.
  5. Goldberger, Arthur S., 1981. "Linear regression after selection," Journal of Econometrics, Elsevier, vol. 15(3), pages 357-366, April.
  6. Heckman, J.J. & Hotz, V.J., 1988. "Choosing Among Alternative Nonexperimental Methods For Estimating The Impact Of Social Programs: The Case Of Manpower Training," University of Chicago - Economics Research Center 88-12, Chicago - Economics Research Center.
  7. Heckman, James J. & Navarro, Salvador, 2003. "Using Matching, Instrumental Variables and Control Functions to Estimate Economic Choice Models," IZA Discussion Papers 768, Institute for the Study of Labor (IZA).
  8. Zhong Zhao, 2004. "Using Matching to Estimate Treatment Effects: Data Requirements, Matching Metrics, and Monte Carlo Evidence," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 91-107, February.
  9. David I. Levine & Gary Painter, 2003. "The Schooling Costs of Teenage Out-of-Wedlock Childbearing: Analysis with a Within-School Propensity-Score-Matching Estimator," The Review of Economics and Statistics, MIT Press, vol. 85(4), pages 884-900, November.
  10. Peter R. Mueser & Carolyn J. Heinrich & Kenneth Troske, 2003. "Welfare to Temporary Work: Implications for Labor Market Outcomes," Working Papers 0308, Department of Economics, University of Missouri.
  11. Joshua D. Angrist, 1995. "Estimating the Labor Market Impact of Voluntary Military Service Using Social Security Data on Military Applicants," NBER Working Papers 5192, National Bureau of Economic Research, Inc.
  12. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2006. "Moving the Goalposts: Addressing Limited Overlap in the Estimation of Average Treatment Effects by Changing the Estimand," NBER Technical Working Papers 0330, National Bureau of Economic Research, Inc.
  13. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, 07.
  14. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Publishing House "SINERGIA PRESS", vol. 31(3), pages 129-137.
  15. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
  16. Chung, Ching-Fan & Goldberger, Arthur S, 1984. "Proportional Projections in Limited Dependent Variable Models," Econometrica, Econometric Society, vol. 52(2), pages 531-34, March.
  17. Ruud, Paul A., 1986. "Consistent estimation of limited dependent variable models despite misspecification of distribution," Journal of Econometrics, Elsevier, vol. 32(1), pages 157-187, June.
  18. Markus Frölich, 2004. "Finite-Sample Properties of Propensity-Score Matching and Weighting Estimators," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 77-90, February.
  19. Ruud, Paul A, 1983. "Sufficient Conditions for the Consistency of Maximum Likelihood Estimation Despite Misspecifications of Distribution in Multinomial Discrete Choice Models," Econometrica, Econometric Society, vol. 51(1), pages 225-28, January.
  20. Rajeev H. Dehejia & Sadek Wahba, 1998. "Causal Effects in Non-Experimental Studies: Re-Evaluating the Evaluation of Training Programs," NBER Working Papers 6586, National Bureau of Economic Research, Inc.
  21. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," NBER Working Papers 6699, National Bureau of Economic Research, Inc.
  22. Dehejia, Rajeev, 2005. "Practical propensity score matching: a reply to Smith and Todd," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 355-364.
  23. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-20, September.
  24. Guildo W. Imbens, 2003. "Sensitivity to Exogeneity Assumptions in Program Evaluation," American Economic Review, American Economic Association, vol. 93(2), pages 126-132, May.
  25. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
  26. repec:oup:restud:v:64:y:1997:i:4:p:605-54 is not listed on IDEAS
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp1873. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Mark Fallak)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.