Advanced Search
MyIDEAS: Login to save this article or follow this journal

Does matching overcome LaLonde's critique of nonexperimental estimators?

Contents:

Author Info

  • A. Smith, Jeffrey
  • E. Todd, Petra

Abstract

This paper applies cross-sectional and longitudinal propensity score matching estimators to data from the National Supported Work (NSW) Demonstration that have been previously analyzed by LaLonde (1986) and Dehejia and Wahba (1999, 2002). We find that estimates of the impact of NSW based on propensity score matching are highly sensitive to both the set of variables included in the scores and the particular analysis sample used in the estimation. Among the estimators we study, the difference-in-differences matching estimator performs the best. We attribute its performance to the fact that it eliminates potential sources of temporally-invariant bias present in the NSW data, such as geographic mismatch between participants and non-participants and the use of a dependent variable measured in different ways for the two groups. Our analysis demonstrates that while propensity score matching is a potentially useful econometric tool, it does not represent a general solution to the evaluation problem.

(This abstract was borrowed from another version of this item.)

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://www.sciencedirect.com/science/article/B6VC0-4CXTY59-1/2/06728bd79899fd9a5b814dfea9fd1560
Download Restriction: Full text for ScienceDirect subscribers only

As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

Bibliographic Info

Article provided by Elsevier in its journal Journal of Econometrics.

Volume (Year): 125 (2005)
Issue (Month): 1-2 ()
Pages: 305-353

as in new window
Handle: RePEc:eee:econom:v:125:y:2005:i:1-2:p:305-353

Contact details of provider:
Web page: http://www.elsevier.com/locate/jeconom

Related research

Keywords:

Other versions of this item:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records," American Economic Review, American Economic Association, vol. 80(3), pages 313-36, June.
  2. Raaum, Oddbjorn & Torp, Hege, 2002. "Labour market training in Norway--effect on earnings," Labour Economics, Elsevier, vol. 9(2), pages 207-247, April.
  3. Lechner, Michael, 1996. "An Evaluation of Public Sector Sponsored Continuous Vocational Training Programs in East Germany," Discussion Papers 539, Institut fuer Volkswirtschaftslehre und Statistik, Abteilung fuer Volkswirtschaftslehre.
  4. Bassi, Laurie J, 1984. "Estimating the Effect of Training Programs with Non-Random Selection," The Review of Economics and Statistics, MIT Press, vol. 66(1), pages 36-43, February.
  5. Joshua D. Angrist, 1998. "Estimating the Labor Market Impact of Voluntary Military Service Using Social Security Data on Military Applicants," Econometrica, Econometric Society, Econometric Society, vol. 66(2), pages 249-288, March.
  6. Rajeev H. Dehejia & Sadek Wahba, 1998. "Propensity Score Matching Methods for Non-experimental Causal Studies," NBER Working Papers 6829, National Bureau of Economic Research, Inc.
  7. David Card & Daniel Sullivan, 1987. "Measuring the Effect of Subsidized Training Programs on Movements In andOut of Employment," NBER Working Papers 2173, National Bureau of Economic Research, Inc.
  8. Heckman, James J, 1979. "Sample Selection Bias as a Specification Error," Econometrica, Econometric Society, Econometric Society, vol. 47(1), pages 153-61, January.
  9. Richard Blundell & Monica Costa Dias, 2000. "Evaluation methods for non-experimental data," Fiscal Studies, Institute for Fiscal Studies, vol. 21(4), pages 427-468, January.
  10. James J. Heckman & Salvador Navarro-Lozano, 2003. "Using Matching, Instrumental Variables and Control Functions to Estimate Economic Choice Models," NBER Working Papers 9497, National Bureau of Economic Research, Inc.
  11. Heckman, James J & Ichimura, Hidehiko & Todd, Petra E, 1997. "Matching as an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," Review of Economic Studies, Wiley Blackwell, Wiley Blackwell, vol. 64(4), pages 605-54, October.
  12. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
  13. Regner, Hakan, 2002. "A nonexperimental evaluation of training programs for the unemployed in Sweden," Labour Economics, Elsevier, vol. 9(2), pages 187-206, April.
  14. James Heckman & Justin L. Tobias & Edward Vytlacil, 2001. "Four Parameters of Interest in the Evaluation of Social Programs," Southern Economic Journal, Southern Economic Association, vol. 68(2), pages 210-223, October.
  15. Heckman, James J. & Robb, Richard Jr., 1985. "Alternative methods for evaluating the impact of interventions : An overview," Journal of Econometrics, Elsevier, Elsevier, vol. 30(1-2), pages 239-267.
  16. Theresa J. Devine & James J. Heckman, 1996. "The Economics of Eligibility Rules for a Social Program: A Study of the Job Training Partnership Act (JTPA)--A Summary Report," Canadian Journal of Economics, Canadian Economics Association, vol. 29(s1), pages 99-104, April.
  17. Couch, Kenneth A, 1992. "New Evidence on the Long-Term Effects of Employment Training Programs," Journal of Labor Economics, University of Chicago Press, University of Chicago Press, vol. 10(4), pages 380-88, October.
  18. James J. Heckman, 1991. "Randomization and Social Policy Evaluation," NBER Technical Working Papers 0107, National Bureau of Economic Research, Inc.
  19. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-20, September.
  20. James J. Heckman, 1995. "Randomization as an Instrumental Variable," NBER Technical Working Papers 0184, National Bureau of Economic Research, Inc.
  21. Eberwein, Curtis & Ham, John C & LaLonde, Robert J, 1997. "The Impact of Being Offered and Receiving Classroom Training on the Employment Histories of Disadvantaged Women: Evidence from Experimental Data," Review of Economic Studies, Wiley Blackwell, Wiley Blackwell, vol. 64(4), pages 655-82, October.
  22. Ashenfelter, Orley & Card, David, 1985. "Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs," The Review of Economics and Statistics, MIT Press, vol. 67(4), pages 648-60, November.
  23. James J. Heckman, 1989. "Choosing Among Alternative Nonexperimental Methods for Estimating the Impact of Social Programs: The Case of Manpower Training," NBER Working Papers 2861, National Bureau of Economic Research, Inc.
  24. James Heckman & Neil Hohmann & Jeffrey Smith & Michael Khoo, 2000. "Substitution And Dropout Bias In Social Experiments: A Study Of An Influential Social Experiment," The Quarterly Journal of Economics, MIT Press, MIT Press, vol. 115(2), pages 651-694, May.
  25. Rajeev H. Dehejia & Sadek Wahba, 1998. "Causal Effects in Non-Experimental Studies: Re-Evaluating the Evaluation of Training Programs," NBER Working Papers 6586, National Bureau of Economic Research, Inc.
  26. Eichler, Martin & Lechner, Michael, 1999. "An Evaluation of Public Employment Programmes in the East German State of Sachsen-Anhalt," IZA Discussion Papers 94, Institute for the Study of Labor (IZA).
  27. Friedlander, Daniel & Robins, Philip K, 1995. "Evaluating Program Evaluations: New Evidence on Commonly Used Nonexperimental Methods," American Economic Review, American Economic Association, vol. 85(4), pages 923-37, September.
  28. Edward Vytlacil & James J. Heckman, 2001. "Policy-Relevant Treatment Effects," American Economic Review, American Economic Association, vol. 91(2), pages 107-111, May.
  29. Michael Lechner, 2005. "Some practical issues in the evaluation of heterogeneous labour market programmes by matching methods," Labor and Demography, EconWPA 0505006, EconWPA.
  30. Lechner, Michael, 1999. "Earnings and Employment Effects of Continuous Off-the-Job Training in East Germany after Unification," Journal of Business & Economic Statistics, American Statistical Association, American Statistical Association, vol. 17(1), pages 74-90, January.
  31. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February.
  32. Joshua Angrist & Jinyong Hahn, 2004. "When to Control for Covariates? Panel Asymptotics for Estimates of Treatment Effects," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 58-72, February.
  33. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," Econometrica, Econometric Society, Econometric Society, vol. 66(5), pages 1017-1098, September.
  34. James J. Heckman, 2001. "Micro Data, Heterogeneity, and the Evaluation of Public Policy: Nobel Lecture," Journal of Political Economy, University of Chicago Press, University of Chicago Press, vol. 109(4), pages 673-748, August.
  35. Zhong Zhao, 2004. "Using Matching to Estimate Treatment Effects: Data Requirements, Matching Metrics, and Monte Carlo Evidence," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 91-107, February.
  36. Heckman, James J & Ichimura, Hidehiko & Todd, Petra, 1998. "Matching as an Econometric Evaluation Estimator," Review of Economic Studies, Wiley Blackwell, Wiley Blackwell, vol. 65(2), pages 261-94, April.
  37. Gary Burtless, 1995. "The Case for Randomized Field Trials in Economic and Policy Research," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 63-84, Spring.
  38. Gary Burtless & Larry L. Orr, 1986. "Are Classical Experiments Needed for Manpower Policy," Journal of Human Resources, University of Wisconsin Press, vol. 21(4), pages 606-639.
  39. Heckman, James J & Smith, Jeffrey A, 1999. "The Pre-programme Earnings Dip and the Determinants of Participation in a Social Programme. Implications for Simple Programme Evaluation Strategies," Economic Journal, Royal Economic Society, Royal Economic Society, vol. 109(457), pages 313-48, July.
  40. Heckman, James J & Smith, Jeffrey, 1997. "Making the Most Out of Programme Evaluations and Social Experiments: Accounting for Heterogeneity in Programme Impacts," Review of Economic Studies, Wiley Blackwell, Wiley Blackwell, vol. 64(4), pages 487-535, October.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
This item has more than 25 citations. To prevent cluttering this page, these citations are listed on a separate page.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:125:y:2005:i:1-2:p:305-353. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.