Advanced Search
MyIDEAS: Login to save this article or follow this journal

The performance of estimators based on the propensity score

Contents:

Author Info

  • Huber, Martin
  • Lechner, Michael
  • Wunsch, Conny

Abstract

We investigate the finite sample properties of a large number of estimators for the average treatment effect on the treated that are suitable when adjustment for observed covariates is required, like inverse probability weighting, kernel and other variants of matching, as well as different parametric models. The simulation design used is based on real data usually employed for the evaluation of labour market programmes in Germany. We vary several dimensions of the design that are of practical importance, like sample size, the type of the outcome variable, and aspects of the selection process. We find that trimming individual observations with too much weight as well as the choice of tuning parameters are important for all estimators. A conclusion from our simulations is that a particular radius matching estimator combined with regression performs best overall, in particular when robustness to misspecifications of the propensity score and different types of outcome variables is considered an important property.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://www.sciencedirect.com/science/article/pii/S0304407613000390
Download Restriction: Full text for ScienceDirect subscribers only

As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

Bibliographic Info

Article provided by Elsevier in its journal Journal of Econometrics.

Volume (Year): 175 (2013)
Issue (Month): 1 ()
Pages: 1-21

as in new window
Handle: RePEc:eee:econom:v:175:y:2013:i:1:p:1-21

Contact details of provider:
Web page: http://www.elsevier.com/locate/jeconom

Related research

Keywords: Propensity score matching; Kernel matching; Inverse probability weighting; Inverse probability tilting; Selection on observables; Empirical Monte Carlo study; Finite sample properties;

Find related papers by JEL classification:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Marco Caliendo & Reinhard Hujer & Stephan L. Thomsen, 2006. "Sectoral Heterogeneity in the Employment Effects of Job Creation Schemes in Germany," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), Justus-Liebig University Giessen, Department of Statistics and Economics, Justus-Liebig University Giessen, Department of Statistics and Economics, vol. 226(2), pages 139-179, March.
  2. Richard Blundell & Monica Costa Dias, 2002. "Alternative approaches to evaluation in empirical microeconomics," CeMMAP working papers, Centre for Microdata Methods and Practice, Institute for Fiscal Studies CWP10/02, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
  3. Blundell, Richard William & Costa Dias, Monica & Meghir, Costas & Van Reenen, John, 2003. "Evaluating the Employment Impact of a Mandatory Job Search Programme," CEPR Discussion Papers 3786, C.E.P.R. Discussion Papers.
  4. Chen, Xiaohong, 2007. "Large Sample Sieve Estimation of Semi-Nonparametric Models," Handbook of Econometrics, Elsevier, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 76 Elsevier.
  5. Lechner, Michael, 1999. "An Evaluation of Public-Sector-Sponsored Continuous Vocational Training Programs in East Germany," IZA Discussion Papers 93, Institute for the Study of Labor (IZA).
  6. Alberto Abadie & Guido W. Imbens, 2009. "Matching on the Estimated Propensity Score," NBER Working Papers 15301, National Bureau of Economic Research, Inc.
  7. Sokbae Lee & Yoon-Jae Whang, 2009. "Nonparametric Tests of Conditional Treatment Effects," Cowles Foundation Discussion Papers 1740, Cowles Foundation for Research in Economics, Yale University.
  8. Joshua D. Angrist, 1995. "Estimating the Labor Market Impact of Voluntary Military Service Using Social Security Data on Military Applicants," NBER Working Papers 5192, National Bureau of Economic Research, Inc.
  9. Miquel, Ruth & Lechner, Michael & Wunsch, Conny, 2005. "Long-Run Effects of Public Sector Sponsored Training in West Germany," ZEW Discussion Papers 05-02, ZEW - Zentrum für Europäische Wirtschaftsforschung / Center for European Economic Research.
  10. John DiNardo & Nicole M. Fortin & Thomas Lemieux, 1995. "Labor Market Institutions and the Distribution of Wages, 1973-1992: A Semiparametric Approach," NBER Working Papers 5093, National Bureau of Economic Research, Inc.
  11. Alberto Abadie & Guido W. Imbens, 2006. "On the Failure of the Bootstrap for Matching Estimators," NBER Technical Working Papers 0325, National Bureau of Economic Research, Inc.
  12. Frölich, Markus, 2002. "Nonparametric IV Estimation of Local Average Treatment Effects with Covariates," IZA Discussion Papers 588, Institute for the Study of Labor (IZA).
  13. Alberto Abadie & Guido W. Imbens, 2006. "Large Sample Properties of Matching Estimators for Average Treatment Effects," Econometrica, Econometric Society, Econometric Society, vol. 74(1), pages 235-267, 01.
  14. Wunsch, Conny & Lechner, Michael, 2007. "What Did All the Money Do? On the General Ineffectiveness of Recent West German Labour Market Programmes," IZA Discussion Papers 2800, Institute for the Study of Labor (IZA).
  15. Rajeev H. Dehejia & Sadek Wahba, 2002. "Propensity score matching methods for non-experimental causal studies," Discussion Papers, Columbia University, Department of Economics 0102-14, Columbia University, Department of Economics.
  16. Jeffrey M. Wooldridge, 2004. "Inverse probability weighted estimation for general missing data problems," CeMMAP working papers, Centre for Microdata Methods and Practice, Institute for Fiscal Studies CWP05/04, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
  17. Huber, Martin & Lechner, Michael & Wunsch, Conny, 2009. "Does Leaving Welfare Improve Health? Evidence for Germany," CEPR Discussion Papers 7421, C.E.P.R. Discussion Papers.
  18. Lechner, Michael & Wunsch, Conny, 2006. "Are Training Programs More Effective When Unemployment Is High?," IZA Discussion Papers 2355, Institute for the Study of Labor (IZA).
  19. Bryan S. Graham & Cristine Campos De Xavier Pinto & Daniel Egel, 2012. "Inverse Probability Tilting for Moment Condition Models with Missing Data," Review of Economic Studies, Oxford University Press, vol. 79(3), pages 1053-1079.
  20. Zhong Zhao, 2004. "Using Matching to Estimate Treatment Effects: Data Requirements, Matching Metrics, and Monte Carlo Evidence," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 91-107, February.
  21. Heckman, James J & Ichimura, Hidehiko & Todd, Petra, 1998. "Matching as an Econometric Evaluation Estimator," Review of Economic Studies, Wiley Blackwell, Wiley Blackwell, vol. 65(2), pages 261-94, April.
  22. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," Econometrica, Econometric Society, Econometric Society, vol. 66(5), pages 1017-1098, September.
  23. Jochen Kluve & Boris Augurzky, 2005. "Assessing the performance of matching algorithms when selection into treatment is strong," RWI Discussion Papers, Rheinisch-Westfälisches Institut für Wirtschaftsforschung 0021, Rheinisch-Westfälisches Institut für Wirtschaftsforschung.
  24. David Card & Jochen Kluve & Andrea Weber, 2010. "Active Labor Market Policy Evaluations: A Meta-Analysis," NBER Working Papers 16173, National Bureau of Economic Research, Inc.
  25. Daniel Millimet & Rusty Tchernis, 2008. "On the Specification of Propensity Scores: with Applications to the Analysis of Trade Policies," Caepr Working Papers, Center for Applied Economics and Policy Research, Economics Department, Indiana University Bloomington 2006-013_Updated, Center for Applied Economics and Policy Research, Economics Department, Indiana University Bloomington.
  26. Caliendo, Marco & Hujer, Reinhard & Thomsen, Stephan L., 2005. "Identifying Effect Heterogeneity to Improve the Effiency of Job Creation Schemes in Germany?," ZEW Discussion Papers 05-21, ZEW - Zentrum für Europäische Wirtschaftsforschung / Center for European Economic Research.
  27. Newey, Whitney K., 1984. "A method of moments interpretation of sequential estimators," Economics Letters, Elsevier, vol. 14(2-3), pages 201-206.
  28. Xiaohong Chen & Han Hong & Alessandro Tarozzi, 2008. "Semiparametric Efficiency in GMM Models of Nonclassical Measurement Errors, Missing Data and Treatment Effects," Cowles Foundation Discussion Papers 1644, Cowles Foundation for Research in Economics, Yale University.
  29. Flores, Carlos A. & Mitnik, Oscar A., 2009. "Evaluating Nonexperimental Estimators for Multiple Treatments: Evidence from Experimental Data," IZA Discussion Papers 4451, Institute for the Study of Labor (IZA).
  30. Guido M. Imbens & Jeffrey M. Wooldridge, 2008. "Recent Developments in the Econometrics of Program Evaluation," NBER Working Papers 14251, National Bureau of Economic Research, Inc.
  31. Hotz, V. Joseph & Crump, Richard K. & Mitnik, Oscar A. & Imbens, Guido, 2009. "Dealing with Limited Overlap in Estimation of Average Treatment Effects," Scholarly Articles 3007645, Harvard University Department of Economics.
  32. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2000. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," NBER Technical Working Papers 0251, National Bureau of Economic Research, Inc.
  33. Busso, Matias & DiNardo, John & McCrary, Justin, 2009. "New Evidence on the Finite Sample Properties of Propensity Score Matching and Reweighting Estimators," IZA Discussion Papers 3998, Institute for the Study of Labor (IZA).
  34. Guido W. Imbens, 2003. "Nonparametric Estimation of Average Treatment Effects under Exogeneity: A Review," NBER Technical Working Papers 0294, National Bureau of Economic Research, Inc.
  35. Reinhard Hujer & Stephan Thomsen & Christopher Zeiss, 2006. "The effects of vocational training programmes on the duration of unemployment in Eastern Germany," AStA Advances in Statistical Analysis, Springer, vol. 90(2), pages 299-321, June.
  36. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-in-Differences Estimates?," The Quarterly Journal of Economics, MIT Press, MIT Press, vol. 119(1), pages 249-275, February.
  37. Markus Frölich, 2004. "Finite-Sample Properties of Propensity-Score Matching and Weighting Estimators," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 77-90, February.
  38. Zhao, Zhong, 2008. "Sensitivity of propensity score methods to the specifications," Economics Letters, Elsevier, vol. 98(3), pages 309-319, March.
  39. Jeffrey Smith & Petra Todd, 2003. "Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators?," University of Western Ontario, CIBC Centre for Human Capital and Productivity Working Papers, University of Western Ontario, CIBC Centre for Human Capital and Productivity 20035, University of Western Ontario, CIBC Centre for Human Capital and Productivity.
  40. Behncke, Stefanie & Frölich, Markus & Lechner, Michael, 2007. "Unemployed and Their Caseworkers: Should They Be Friends or Foes?," CEPR Discussion Papers 6558, C.E.P.R. Discussion Papers.
  41. Jinyong Hahn, 1998. "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica, Econometric Society, Econometric Society, vol. 66(2), pages 315-332, March.
  42. Lechner, Michael, 1999. "Earnings and Employment Effects of Continuous Off-the-Job Training in East Germany after Unification," Journal of Business & Economic Statistics, American Statistical Association, American Statistical Association, vol. 17(1), pages 74-90, January.
  43. Gerfin, Michael & Lechner, Michael, 2000. "Microeconometric Evaluation of the Active Labour Market Policy in Switzerland," IZA Discussion Papers 154, Institute for the Study of Labor (IZA).
  44. Michael LECHNER, 2008. "A Note on the Common Support Problem in Applied Evaluation Studies," Annales d'Economie et de Statistique, ENSAE, issue 91-92, pages 217-235.
  45. Ben B. Hansen, 2004. "Full Matching in an Observational Study of Coaching for the SAT," Journal of the American Statistical Association, American Statistical Association, American Statistical Association, vol. 99, pages 609-618, January.
  46. Patrick Kline, 2011. "Oaxaca-Blinder as a Reweighting Estimator," American Economic Review, American Economic Association, vol. 101(3), pages 532-37, May.
  47. Jose C. GALDO & Jeffrey SMITH & Dan BLACK, 2008. "Bandwidth Selection and the Estimation of Treatment Effects with Unbalanced Data," Annales d'Economie et de Statistique, ENSAE, issue 91-92, pages 189-216.
  48. Stefanie Behncke & Markus Frölich & Michael Lechner, 2010. "A Caseworker Like Me - Does The Similarity Between The Unemployed and Their Caseworkers Increase Job Placements?," Economic Journal, Royal Economic Society, Royal Economic Society, vol. 120(549), pages 1430-1459, December.
  49. Peter Hall & Jeff Racine & Qi Li, 2004. "Cross-Validation and the Estimation of Conditional Probability Densities," Journal of the American Statistical Association, American Statistical Association, American Statistical Association, vol. 99, pages 1015-1026, December.
  50. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-20, September.
  51. Zhao, Zhong, 2006. "Matching Estimators and the Data from the National Supported Work Demonstration Again," IZA Discussion Papers 2375, Institute for the Study of Labor (IZA).
  52. Alberto Abadie & Guido W. Imbens, 2002. "Simple and Bias-Corrected Matching Estimators for Average Treatment Effects," NBER Technical Working Papers 0283, National Bureau of Economic Research, Inc.
  53. Alberto Abadie, 2005. "Semiparametric Difference-in-Differences Estimators," Review of Economic Studies, Oxford University Press, vol. 72(1), pages 1-19.
  54. Hujer, Reinhard & Thomsen, Stephan L., 2010. "How do the employment effects of job creation schemes differ with respect to the foregoing unemployment duration?," Labour Economics, Elsevier, vol. 17(1), pages 38-51, January.
  55. Hujer, Reinhard & Caliendo, Marco & Thomsen, Stephan L., 2003. "New Evidence on the Effects of Job Creation Schemes in Germany - A Matching Approach with Threefold Heterogeneity," IZA Discussion Papers 750, Institute for the Study of Labor (IZA).
  56. Martin Huber, 2010. "Testing for covariate balance using quantile regression and resampling methods," University of St. Gallen Department of Economics working paper series 2010, Department of Economics, University of St. Gallen 2010-18, Department of Economics, University of St. Gallen.
  57. Bryan S. Graham & Cristine Campos de Xavier Pinto & Daniel Egel, 2011. "Efficient Estimation of Data Combination Models by the Method of Auxiliary-to-Study Tilting (AST)," NBER Working Papers 16928, National Bureau of Economic Research, Inc.
  58. James J. Heckman, 1976. "The Common Structure of Statistical Models of Truncation, Sample Selection and Limited Dependent Variables and a Simple Estimator for Such Models," NBER Chapters, in: Annals of Economic and Social Measurement, Volume 5, number 4, pages 475-492 National Bureau of Economic Research, Inc.
  59. Lechner, Michael, 2009. "Long-run labour market and health effects of individual sports activities," Journal of Health Economics, Elsevier, vol. 28(4), pages 839-854, July.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
This item has more than 25 citations. To prevent cluttering this page, these citations are listed on a separate page.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:175:y:2013:i:1:p:1-21. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.