This file is part of IDEAS, which uses RePEc data


[ Papers | Articles | Software | Books | Chapters | Authors | Institutions | JEL Classification | NEP reports | Search | New papers by email | Author registration | Rankings | Volunteers | FAQ | Blog | Help! ]

Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators?

Author info | Abstract | Publisher info | Download info | Related research | Statistics
Author Info
Jeffrey Smith (University of Michigan)
Petra Todd

Additional information is available for the following registered author(s):

Abstract

This paper applies cross-sectional and longitudinal propensity score matching estimators to data from the National Supported Work (NSW) Demonstration that have been previously analyzed by LaLonde (1986) and Dehejia and Wahba (1999, 2002). We find that estimates of the impact of NSW based on propensity score matching are highly sensitive to both the set of variables included in the scores and the particular analysis sample used in the estimation. Among the estimators we study, the difference-in-differences matching estimator performs the best. We attribute its performance to the fact that it eliminates potential sources of temporally-invariant bias present in the NSW data, such as geographic mismatch between participants and non-participants and the use of a dependent variable measured in different ways for the two groups. Our analysis demonstrates that while propensity score matching is a potentially useful econometric tool, it does not represent a general solution to the evaluation problem.

Download Info
To download:

If you experience problems downloading a file, check if you have the proper application to view it first. Information about this may be contained in the File-Format links below. In case of further problems read the IDEAS help file. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://economics.uwo.ca/centres/cibc/wp2003/Smith05.pdf
File Format: application/pdf
File Function:
Download Restriction: no

Publisher Info
Paper provided by University of Western Ontario, CIBC Human Capital and Productivity Project in its series University of Western Ontario, CIBC Human Capital and Productivity Project Working Papers with number 20035.

Download reference. The following formats are available: HTML, plain text, BibTeX, RIS (EndNote), ReDIF
Length:
Date of creation: 2003
Date of revision:
Handle: RePEc:uwo:hcuwoc:20035

Contact details of provider:
Postal: CIBC Human Capital and Productivity Project, Social Science Centre, University of Western Ontario, London, Ontario, Canada N6A 5C2
Phone: 519-661-2111 Ext.85228
Web page: http://economics.uwo.ca/econref/WorkingPapers/

For technical questions regarding this item, or to correct its listing, contact: ().

Related research
Keywords:

Other versions of this item:

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
  1. Eberwein, Curtis & Ham, John C & LaLonde, Robert J, 1997. "The Impact of Being Offered and Receiving Classroom Training on the Employment Histories of Disadvantaged Women: Evidence from Experimental Data," Review of Economic Studies, Blackwell Publishing, vol. 64(4), pages 655-82, October. [Downloadable!] (restricted)
  2. Regner, Hakan, 2002. "A nonexperimental evaluation of training programs for the unemployed in Sweden," Labour Economics, Elsevier, vol. 9(2), pages 187-206, April. [Downloadable!] (restricted)
  3. Heckman, J J & Tobias, Justin & Vytlacil, Ed, 2004. "Four Parameters of Interest in the Evaluation of Social Programs," Staff General Research Papers 12022, Iowa State University, Department of Economics.
  4. James Heckman & Neil Hohmann & Jeffrey Smith, 1998. "Substitution and Dropout Bias in Social Experiments: A Study of an Influential Social Experiment," UWO Department of Economics Working Papers 9819, University of Western Ontario, Department of Economics. [Downloadable!]
    Other versions:
  5. Michael Lechner, 2005. "Some practical issues in the evaluation of heterogeneous labour market programmes by matching methods," Labor and Demography 0505006, EconWPA. [Downloadable!]
    Other versions:
  6. James J. Heckman, 2001. "Micro Data, Heterogeneity, and the Evaluation of Public Policy: Nobel Lecture," Journal of Political Economy, University of Chicago Press, vol. 109(4), pages 673-748, August. [Downloadable!] (restricted)
  7. James Heckman & Salvador Navarro-Lozano, 2004. "Using Matching, Instrumental Variables, and Control Functions to Estimate Economic Choice Models," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 30-57, 08. [Downloadable!] (restricted)
    Other versions:
  8. Heckman, James J, 1979. "Sample Selection Bias as a Specification Error," Econometrica, Econometric Society, vol. 47(1), pages 153-61, January. [Downloadable!] (restricted)
  9. Heckman, James J & Smith, Jeffrey A, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring. [Downloadable!] (restricted)
  10. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records," American Economic Review, American Economic Association, vol. 80(3), pages 313-36, June.
    Other versions:
  11. James J. Heckman, 1995. "Randomization as an Instrumental Variable," NBER Technical Working Papers 0184, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
  12. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," Econometrica, Econometric Society, vol. 66(5), pages 1017-1098, September.
    Other versions:
  13. James J. Heckman, 1991. "Randomization and Social Policy Evaluation," NBER Technical Working Papers 0107, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
  14. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-20, September. [Downloadable!] (restricted)
  15. Rajeev H. Dehejia & Sadek Wahba, 2002. "Propensity Score-Matching Methods For Nonexperimental Causal Studies," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 151-161, February. [Downloadable!] (restricted)
    Other versions:
  16. Heckman, James J. & Robb, Richard Jr., 1985. "Alternative methods for evaluating the impact of interventions : An overview," Journal of Econometrics, Elsevier, vol. 30(1-2), pages 239-267. [Downloadable!] (restricted)
  17. Eichler, Martin & Lechner, Michael, 2002. "An evaluation of public employment programmes in the East German State of Sachsen-Anhalt," Labour Economics, Elsevier, vol. 9(2), pages 143-186, April. [Downloadable!] (restricted)
    Other versions:
  18. James J. Heckman & Edward Vytlacil, 2001. "Policy-Relevant Treatment Effects," American Economic Review, American Economic Association, vol. 91(2), pages 107-111, May. [Downloadable!] (restricted)
  19. Zhong Zhao, 2004. "Using Matching to Estimate Treatment Effects: Data Requirements, Matching Metrics, and Monte Carlo Evidence," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 91-107, 08. [Downloadable!] (restricted)
  20. Bassi, Laurie J, 1984. "Estimating the Effect of Training Programs with Non-Random Selection," The Review of Economics and Statistics, MIT Press, vol. 66(1), pages 36-43, February. [Downloadable!] (restricted)
  21. Lechner, Michael, 1999. "Earnings and Employment Effects of Continuous Off-the-Job Training in East Germany after Unification," Journal of Business & Economic Statistics, American Statistical Association, vol. 17(1), pages 74-90, January.
  22. Heckman, James J & Ichimura, Hidehiko & Todd, Petra, 1998. "Matching as an Econometric Evaluation Estimator," Review of Economic Studies, Blackwell Publishing, vol. 65(2), pages 261-94, April. [Downloadable!] (restricted)
  23. Heckman, James J & Ichimura, Hidehiko & Todd, Petra E, 1997. "Matching as an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," Review of Economic Studies, Blackwell Publishing, vol. 64(4), pages 605-54, October. [Downloadable!] (restricted)
  24. Card, David & Sullivan, Daniel G, 1988. "Measuring the Effect of Subsidized Training Programs on Movements in and out of Employment," Econometrica, Econometric Society, vol. 56(3), pages 497-530, May. [Downloadable!] (restricted)
    Other versions:
  25. Rajeev H. Dehejia & Sadek Wahba, 1998. "Causal Effects in Non-Experimental Studies: Re-Evaluating the Evaluation of Training Programs," NBER Working Papers 6586, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
  26. Couch, Kenneth A, 1992. "New Evidence on the Long-Term Effects of Employment Training Programs," Journal of Labor Economics, University of Chicago Press, vol. 10(4), pages 380-88, October. [Downloadable!] (restricted)
  27. Burtless, Gary, 1995. "The Case for Randomized Field Trials in Economic and Policy Research," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 63-84, Spring. [Downloadable!] (restricted)
  28. Joshua D. Angrist, 1998. "Estimating the Labor Market Impact of Voluntary Military Service Using Social Security Data on Military Applicants," Econometrica, Econometric Society, vol. 66(2), pages 249-288, March.
    Other versions:
  29. James J. Heckman, 1989. "Choosing Among Alternative Nonexperimental Methods for Estimating the Impact of Social Programs: The Case of Manpower Training," NBER Working Papers 2861, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
    Other versions:
  30. Heckman, James J & Smith, Jeffrey A, 1999. "The Pre-programme Earnings Dip and the Determinants of Participation in a Social Programme. Implications for Simple Programme Evaluation Strategies," Economic Journal, Royal Economic Society, vol. 109(457), pages 313-48, July. [Downloadable!] (restricted)
  31. Theresa J. Devine & James J. Heckman, 1996. "The Economics of Eligibility Rules for a Social Program: A Study of the Job Training Partnership Act (JTPA)--A Summary Report," Canadian Journal of Economics, Canadian Economics Association, vol. 29(s1), pages 99-104, April.
  32. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February. [Downloadable!] (restricted)
  33. Michael Lechner, 1999. "An Evaluation of Public-Sector-Sponsored Continuous Vocational Training Programs in East Germany," IZA Discussion Papers 93, Institute for the Study of Labor (IZA). [Downloadable!]
    Other versions:
  34. Ashenfelter, Orley & Card, David, 1985. "Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs," The Review of Economics and Statistics, MIT Press, vol. 67(4), pages 648-60, November. [Downloadable!] (restricted)
    Other versions:
  35. Joshua Angrist & Jinyong Hahn, 2004. "When to Control for Covariates? Panel Asymptotics for Estimates of Treatment Effects," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 58-72, 06. [Downloadable!] (restricted)
  36. Friedlander, Daniel & Robins, Philip K, 1995. "Evaluating Program Evaluations: New Evidence on Commonly Used Nonexperimental Methods," American Economic Review, American Economic Association, vol. 85(4), pages 923-37, September. [Downloadable!] (restricted)
Full references

Cited by:
(explanations, Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.)
This item has more than 25 citations. To prevent cluttering this page, these citations are listed on a separate page.
Statistics
Access and download statistics

Did you know? A tutorial is available.

This page was last updated on 2008-9-24.

This information is provided to you by IDEAS at the Department of Economics, College of Liberal Arts and Sciences, University of Connecticut using RePEc data on a server sponsored by the Society for Economic Dynamics.