This file is part of IDEAS, which uses RePEc data


[ Papers | Articles | Software | Books | Chapters | Authors | Institutions | JEL Classification | NEP reports | Search | New papers by email | Author registration | Rankings | Volunteers | FAQ | Blog | Help! ]

Sensitivity of Propensity Score Methods to the Specifications

Author info | Abstract | Publisher info | Download info | Related research | Statistics
Author Info
Zhong Zhao () (IZA Bonn)

Additional information is available for the following registered author(s):

Abstract

Propensity score matching estimators have two advantages. One is that they overcome the curse of dimensionality of covariate matching, and the other is that they are nonparametric. However, the propensity score is usually unknown and needs to be estimated. If we estimate it nonparametrically, we are incurring the curse-of-dimensionality problem we are trying to avoid. If we estimate it parametrically, how sensitive the estimated treatment effects are to the specifications of the propensity score becomes an important question. In this paper, we study this issue. First, we use a Monte Carlo experimental method to investigate the sensitivity issue under the unconfoundedness assumption. We find that the estimates are not sensitive to the specifications. Next, we provide some theoretical justifications, using the insight from Rosenbaum and Rubin (1983) that any score finer than the propensity score is a balancing score. Then, we reconcile our finding with the finding in Smith and Todd (2005) that, if the unconfoundedness assumption fails, the matching results can be sensitive. However, failure of the unconfoundedness assumption will not necessarily result in sensitive estimates. Matching estimators can be speciously robust in the sense that the treatment effects are consistently overestimated or underestimated. Sensitivity checks applied in empirical studies are helpful in eliminating sensitive cases, but in general, it cannot help to solve the fundamental problem that the matching assumptions are inherently untestable. Last, our results suggest that including irrelevant variables in the propensity score will not bias the results, but overspecifying it (e.g., adding unnecessary nonlinear terms) probably will.

Download Info
To download:

If you experience problems downloading a file, check if you have the proper application to view it first. Information about this may be contained in the File-Format links below. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: ftp://repec.iza.org/RePEc/Discussionpaper/dp1873.pdf
File Format: application/pdf
File Function:
Download Restriction: no

Publisher Info
Paper provided by Institute for the Study of Labor (IZA) in its series IZA Discussion Papers with number 1873.

Download reference. The following formats are available: HTML (with abstract), plain text (with abstract), BibTeX, RIS (EndNote, RefMan, ProCite), ReDIF
Length: 37 pages
Date of creation: Dec 2005
Date of revision:
Handle: RePEc:iza:izadps:dp1873

Contact details of provider:
Postal: IZA, P.O. Box 7240, D-53072 Bonn, Germany
Phone: +49 228 3894 223
Fax: +49 228 3894 180
Web page: http://www.iza.org

Order Information:
Postal: IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany
Email:

For technical questions regarding this item, or to correct its listing, contact: (Mark Fallak).

Related research
Keywords: sensitivity; propensity score; matching; causal model; Monte Carlo;

Other versions of this item:

Find related papers by JEL classification:
C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: General - - - Semiparametric and Nonparametric Methods
C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: General - - - Statistical Simulation Methods
C16 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: General - - - Econometric and Statistical Methods; Specific Distributions
C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation and Testing

This paper has been announced in the following NEP Reports:

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
  1. Guido W. Imbens, 1999. "The Role of the Propensity Score in Estimating Dose-Response Functions," NBER Technical Working Papers 0237, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
  2. Ruud, Paul A., 1986. "Consistent estimation of limited dependent variable models despite misspecification of distribution," Journal of Econometrics, Elsevier, vol. 32(1), pages 157-187, June. [Downloadable!] (restricted)
  3. James Heckman & Salvador Navarro-Lozano, 2004. "Using Matching, Instrumental Variables, and Control Functions to Estimate Economic Choice Models," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 30-57, 08. [Downloadable!] (restricted)
    Other versions:
  4. Heckman, James J, 1979. "Sample Selection Bias as a Specification Error," Econometrica, Econometric Society, vol. 47(1), pages 153-61, January. [Downloadable!] (restricted)
  5. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," Econometrica, Econometric Society, vol. 66(5), pages 1017-1098, September.
    Other versions:
  6. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-20, September. [Downloadable!] (restricted)
  7. James J. Heckman & Edward Vytlacil, 2005. "Structural Equations, Treatment Effects and Econometric Policy Evaluation," NBER Working Papers 11259, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
    Other versions:
  8. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, 07. [Downloadable!] (restricted)
    Other versions:
  9. Zhong Zhao, 2004. "Using Matching to Estimate Treatment Effects: Data Requirements, Matching Metrics, and Monte Carlo Evidence," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 91-107, 08. [Downloadable!] (restricted)
  10. Goldberger, Arthur S., 1981. "Linear regression after selection," Journal of Econometrics, Elsevier, vol. 15(3), pages 357-366, April. [Downloadable!] (restricted)
  11. Guildo W. Imbens, 2003. "Sensitivity to Exogeneity Assumptions in Program Evaluation," American Economic Review, American Economic Association, vol. 93(2), pages 126-132, May. [Downloadable!]
  12. Heckman, James J & Ichimura, Hidehiko & Todd, Petra E, 1997. "Matching as an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," Review of Economic Studies, Blackwell Publishing, vol. 64(4), pages 605-54, October. [Downloadable!] (restricted)
  13. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353. [Downloadable!] (restricted)
    Other versions:
  14. David I. Levine & Gary Painter, 2003. "The Schooling Costs of Teenage Out-of-Wedlock Childbearing: Analysis with a Within-School Propensity-Score-Matching Estimator," The Review of Economics and Statistics, MIT Press, vol. 85(4), pages 884-900, 06. [Downloadable!] (restricted)
  15. Rajeev H. Dehejia & Sadek Wahba, 1998. "Causal Effects in Non-Experimental Studies: Re-Evaluating the Evaluation of Training Programs," NBER Working Papers 6586, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
  16. Chung, Ching-Fan & Goldberger, Arthur S, 1984. "Proportional Projections in Limited Dependent Variable Models," Econometrica, Econometric Society, vol. 52(2), pages 531-34, March. [Downloadable!] (restricted)
  17. Joshua D. Angrist, 1998. "Estimating the Labor Market Impact of Voluntary Military Service Using Social Security Data on Military Applicants," Econometrica, Econometric Society, vol. 66(2), pages 249-288, March.
    Other versions:
  18. Dehejia, Rajeev, 2005. "Practical propensity score matching: a reply to Smith and Todd," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 355-364. [Downloadable!] (restricted)
  19. Ruud, Paul A, 1983. "Sufficient Conditions for the Consistency of Maximum Likelihood Estimation Despite Misspecifications of Distribution in Multinomial Discrete Choice Models," Econometrica, Econometric Society, vol. 51(1), pages 225-28, January. [Downloadable!] (restricted)
  20. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2006. "Moving the Goalposts: Addressing Limited Overlap in the Estimation of Average Treatment Effects by Changing the Estimand," NBER Technical Working Papers 0330, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
  21. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, 06. [Downloadable!] (restricted)
    Other versions:
  22. Carolyn J. Heinrich & Peter R. Mueser & Kenneth R. Troske, 2005. "Welfare to Temporary Work: Implications for Labor Market Outcomes," The Review of Economics and Statistics, MIT Press, vol. 87(1), pages 154-173, December. [Downloadable!] (restricted)
    Other versions:
  23. Jinyong Hahn, 1998. "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica, Econometric Society, vol. 66(2), pages 315-332, March.
  24. Markus Frölich, 2004. "Finite-Sample Properties of Propensity-Score Matching and Weighting Estimators," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 77-90, 07. [Downloadable!] (restricted)
  25. Peter R. Mueser & Kenneth R. Troske & Alexey Gorislavsky, 2006. "Using State Administrative Data to Measure Program Performance," Working Papers 0702, Department of Economics, University of Missouri. [Downloadable!]
    Other versions:
Full references

Cited by:
(explanations, Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.)

  1. Arpino, Bruno & Mealli, Fabrizia, 2008. "The specification of the propensity score in multilevel observational studies," MPRA Paper 17407, University Library of Munich, Germany. [Downloadable!]
  2. Michael Lechner & Blaise Melly, 2007. "Earnings Effects of Training Programs," University of St. Gallen Department of Economics working paper series 2007 2007-28, Department of Economics, University of St. Gallen. [Downloadable!]
    Other versions:
  3. Janet Currie & Erdal Tekin, 2006. "Does Child Abuse Cause Crime?," NBER Working Papers 12171, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
    Other versions:
  4. Daniel Millimet & Rusty Tchernis, 2006. "On the Specification of Propensity Scores: with an Application to the WTO-Environment Debate," Caepr Working Papers 2006-013, Center for Applied Economics and Policy Research, Economics Department, Indiana University Bloomington. [Downloadable!]
Statistics
Access and download statistics

Did you know? You can create a compilation of all publications of a group of people, say alumni of a program, your students or memers of an association.

This page was last updated on 2009-11-30.

This information is provided to you by IDEAS at the Department of Economics, College of Liberal Arts and Sciences, University of Connecticut using RePEc data on a server sponsored by the Society for Economic Dynamics.