IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Login to save this paper or follow this series

How to Control for Many Covariates? Reliable Estimators Based on the Propensity Score

  • Huber, Martin

    ()

    (University of Fribourg)

  • Lechner, Michael

    ()

    (University of St. Gallen)

  • Wunsch, Conny

    ()

    (University of Basel)

We investigate the finite sample properties of a large number of estimators for the average treatment effect on the treated that are suitable when adjustment for observable covariates is required, like inverse probability weighting, kernel and other variants of matching, as well as different parametric models. The simulation design used is based on real data usually employed for the evaluation of labour market programmes in Germany. We vary several dimensions of the design that are of practical importance, like sample size, the type of the outcome variable, and aspects of the selection process. We find that trimming individual observations with too much weight as well as the choice of tuning parameters is important for all estimators. The key conclusion from our simulations is that a particular radius matching estimator combined with regression performs best overall, in particular when robustness to misspecifications of the propensity score is considered an important property.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://ftp.iza.org/dp5268.pdf
Download Restriction: no

Paper provided by Institute for the Study of Labor (IZA) in its series IZA Discussion Papers with number 5268.

as
in new window

Length: 65 pages
Date of creation: Oct 2010
Date of revision:
Publication status: published as 'The Performance of Estimators Based on the Propensity Score' in: Journal of Econometrics, 2013, 175 (1), 1-21
Handle: RePEc:iza:izadps:dp5268
Contact details of provider: Postal: IZA, P.O. Box 7240, D-53072 Bonn, Germany
Phone: +49 228 3894 223
Fax: +49 228 3894 180
Web page: http://www.iza.org

Order Information: Postal: IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany
Email:


References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Richard Blundell & Monica Costa Dias & Costas Meghir & John Van Reenen, 2004. "Evaluating the Employment Impact of a Mandatory Job Search Program," Journal of the European Economic Association, MIT Press, vol. 2(4), pages 569-606, 06.
  2. Dehejia, Rajeev, 2005. "Practical propensity score matching: a reply to Smith and Todd," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 355-364.
  3. Richard Blundell & Monica Costa Dias, 2002. "Alternative approaches to evaluation in empirical microeconomics," CeMMAP working papers CWP10/02, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
  4. Markus Froelich, 2002. "Nonparametric IV estimation of local average treatment effects with covariates," University of St. Gallen Department of Economics working paper series 2002 2002-19, Department of Economics, University of St. Gallen.
  5. Imbens, Guido W. & Wooldridge, Jeffrey M., 2008. "Recent Developments in the Econometrics of Program Evaluation," IZA Discussion Papers 3640, Institute for the Study of Labor (IZA).
  6. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, 07.
  7. Markus Frölich, 2004. "Finite-Sample Properties of Propensity-Score Matching and Weighting Estimators," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 77-90, February.
  8. Conny Wunsch & Michael Lechner, 2008. "What Did All the Money Do? On the General Ineffectiveness of Recent West German Labour Market Programmes," Kyklos, Wiley Blackwell, vol. 61(1), pages 134-174, 02.
  9. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-20, September.
  10. Zhong Zhao, 2004. "Using Matching to Estimate Treatment Effects: Data Requirements, Matching Metrics, and Monte Carlo Evidence," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 91-107, February.
  11. Newey, Whitney K., 1984. "A method of moments interpretation of sequential estimators," Economics Letters, Elsevier, vol. 14(2-3), pages 201-206.
  12. Hujer, Reinhard & Caliendo, Marco & Thomsen, Stephan L., 2004. "New evidence on the effects of job creation schemes in Germany--a matching approach with threefold heterogeneity," Research in Economics, Elsevier, vol. 58(4), pages 257-302, December.
  13. Carlos A. Flores & Oscar A. Mitnik, 2009. "Evaluating Nonexperimental Estimators for Multiple Treatments: Evidence from Experimental Data," Working Papers 2010-10, University of Miami, Department of Economics.
  14. Behncke, Stefanie & Frölich, Markus & Lechner, Michael, 2008. "A Caseworker Like Me - Does the Similarity between unemployed and Caseworker Increase Job Placements?," CEPR Discussion Papers 6784, C.E.P.R. Discussion Papers.
  15. Zhao, Zhong, 2008. "Sensitivity of propensity score methods to the specifications," Economics Letters, Elsevier, vol. 98(3), pages 309-319, March.
  16. Lechner, Michael, 1999. "Earnings and Employment Effects of Continuous Off-the-Job Training in East Germany after Unification," Journal of Business & Economic Statistics, American Statistical Association, vol. 17(1), pages 74-90, January.
  17. Daniel Millimet & Rusty Tchernis, 2008. "On the Specification of Propensity Scores: with Applications to the Analysis of Trade Policies," Caepr Working Papers 2006-013_Updated, Center for Applied Economics and Policy Research, Economics Department, Indiana University Bloomington.
  18. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-in-Differences Estimates?," The Quarterly Journal of Economics, MIT Press, vol. 119(1), pages 249-275, February.
  19. Heckman, James J & Ichimura, Hidehiko & Todd, Petra, 1998. "Matching as an Econometric Evaluation Estimator," Review of Economic Studies, Wiley Blackwell, vol. 65(2), pages 261-94, April.
  20. Galdo, Jose C. & Smith, Jeffrey A. & Black, Dan A., 2007. "Bandwidth Selection and the Estimation of Treatment Effects with Unbalanced Data," IZA Discussion Papers 3095, Institute for the Study of Labor (IZA).
  21. Jeffrey M. Wooldridge, 2004. "Inverse probability weighted estimation for general missing data problems," CeMMAP working papers CWP05/04, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
  22. Nidardo, J. & Fortin, N. & Lemieux, T., 1994. "Labor Market Institutions and the Distribution of Wages, 1973-1992: A Semiparametric Approach," Papers 93-94-15, California Irvine - School of Social Sciences.
  23. Jeffrey Smith & Petra Todd, 2003. "Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators?," University of Western Ontario, CIBC Centre for Human Capital and Productivity Working Papers 20035, University of Western Ontario, CIBC Centre for Human Capital and Productivity.
  24. Marco Caliendo & Reinhard Hujer & Stephan L. Thomsen, 2006. "Sectoral Heterogeneity in the Employment Effects of Job Creation Schemes in Germany," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), Justus-Liebig University Giessen, Department of Statistics and Economics, vol. 226(2), pages 139-179, March.
  25. Lechner, Michael, 1996. "An Evaluation of Public Sector Sponsored Continuous Vocational Training Programs in East Germany," Discussion Papers 539, Institut fuer Volkswirtschaftslehre und Statistik, Abteilung fuer Volkswirtschaftslehre.
  26. Jochen Kluve & Boris Augurzky, 2007. "Assessing the performance of matching algorithms when selection into treatment is strong," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(3), pages 533-557.
  27. Guido W. Imbens, 2003. "Nonparametric Estimation of Average Treatment Effects under Exogeneity: A Review," NBER Technical Working Papers 0294, National Bureau of Economic Research, Inc.
  28. Ahmed Khwaja & Gabriel Picone & Martin Salm & Justin G. Trogdon, 2011. "A comparison of treatment effects estimators using a structural model of AMI treatment choices and severity of illness information from hospital charts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(5), pages 825-853, 08.
  29. Martin Huber, 2011. "Testing for covariate balance using quantile regression and resampling methods," Journal of Applied Statistics, Taylor & Francis Journals, vol. 38(12), pages 2881-2899, February.
  30. Alberto Abadie, 2005. "Semiparametric Difference-in-Differences Estimators," Review of Economic Studies, Oxford University Press, vol. 72(1), pages 1-19.
  31. James J. Heckman, 1976. "The Common Structure of Statistical Models of Truncation, Sample Selection and Limited Dependent Variables and a Simple Estimator for Such Models," NBER Chapters, in: Annals of Economic and Social Measurement, Volume 5, number 4, pages 475-492 National Bureau of Economic Research, Inc.
  32. Zhao, Zhong, 2006. "Matching Estimators and the Data from the National Supported Work Demonstration Again," IZA Discussion Papers 2375, Institute for the Study of Labor (IZA).
  33. Rajeev H. Dehejia & Sadek Wahba, 1998. "Propensity Score Matching Methods for Non-experimental Causal Studies," NBER Working Papers 6829, National Bureau of Economic Research, Inc.
  34. Michael Gerfin & Michael Lechner, 2002. "A Microeconometric Evaluation of the Active Labour Market Policy in Switzerland," Economic Journal, Royal Economic Society, vol. 112(482), pages 854-893, October.
  35. Alberto Abadie & Guido W. Imbens, 2009. "Matching on the Estimated Propensity Score," NBER Working Papers 15301, National Bureau of Economic Research, Inc.
  36. Hujer, Reinhard & Thomsen, Stephan L., 2010. "How do the employment effects of job creation schemes differ with respect to the foregoing unemployment duration?," Labour Economics, Elsevier, vol. 17(1), pages 38-51, January.
  37. Behncke, Stefanie & Frölich, Markus & Lechner, Michael, 2007. "Unemployed and Their Caseworkers: Should They Be Friends or Foes?," CEPR Discussion Papers 6558, C.E.P.R. Discussion Papers.
  38. Hotz, V. Joseph & Crump, Richard K. & Mitnik, Oscar A. & Imbens, Guido, 2009. "Dealing with Limited Overlap in Estimation of Average Treatment Effects," Scholarly Articles 3007645, Harvard University Department of Economics.
  39. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," Econometrica, Econometric Society, vol. 66(5), pages 1017-1098, September.
  40. Jinyong Hahn, 1998. "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica, Econometric Society, vol. 66(2), pages 315-332, March.
  41. Peter Hall & Jeff Racine & Qi Li, 2004. "Cross-Validation and the Estimation of Conditional Probability Densities," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 1015-1026, December.
  42. Lechner, Michael, 2009. "Long-run labour market and health effects of individual sports activities," Journal of Health Economics, Elsevier, vol. 28(4), pages 839-854, July.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp5268. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Mark Fallak)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.