Advanced Search
MyIDEAS: Login to save this paper or follow this series

Using State Administrative Data to Measure Program Performance

Contents:

Author Info

  • Mueser, Peter R.

    ()
    (University of Missouri-Columbia)

  • Troske, Kenneth

    ()
    (University of Kentucky)

  • Gorislavsky, Alexey

    ()
    (University of Missouri-Columbia)

Abstract

This paper uses administrative data from Missouri to examine the sensitivity of job training program impact estimates based on alternative nonexperimental methods. In addition to simple regression adjustment, we consider Mahalanobis distance matching and a variety of methods using propensity score matching. In each case, we consider estimates based on levels of post-program earnings as well as difference-in-difference estimates based on comparison of pre- and post-program earnings. Specification tests suggest that the difference-in-difference estimator may provide a better measure of program impact. We find that propensity score matching is generally most effective, but the detailed implementation of the method is not of critical importance. Our analyses demonstrate that existing data available at the state level can be used to obtain useful estimates of program impact.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://ftp.iza.org/dp786.pdf
Download Restriction: no

Bibliographic Info

Paper provided by Institute for the Study of Labor (IZA) in its series IZA Discussion Papers with number 786.

as in new window
Length: 62 pages
Date of creation: May 2003
Date of revision:
Handle: RePEc:iza:izadps:dp786

Contact details of provider:
Postal: IZA, P.O. Box 7240, D-53072 Bonn, Germany
Phone: +49 228 3894 223
Fax: +49 228 3894 180
Web page: http://www.iza.org

Order Information:
Postal: IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany
Email:

Related research

Keywords: program evaluation; matching; job training;

Other versions of this item:

Find related papers by JEL classification:

This paper has been announced in the following NEP Reports:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Charles F. Manski, 1996. "Learning about Treatment Effects from Experiments with Random Assignment of Treatments," Journal of Human Resources, University of Wisconsin Press, vol. 31(4), pages 709-733.
  2. Friedlander, Daniel & Robins, Philip K, 1995. "Evaluating Program Evaluations: New Evidence on Commonly Used Nonexperimental Methods," American Economic Review, American Economic Association, American Economic Association, vol. 85(4), pages 923-37, September.
  3. Heckman, James J & Ichimura, Hidehiko & Todd, Petra, 1998. "Matching as an Econometric Evaluation Estimator," Review of Economic Studies, Wiley Blackwell, Wiley Blackwell, vol. 65(2), pages 261-94, April.
  4. James Heckman & Neil Hohmann & Jeffrey Smith, 1998. "Substitution and Dropout Bias in Social Experiments: A Study of an Influential Social Experiment," UWO Department of Economics Working Papers, University of Western Ontario, Department of Economics 9819, University of Western Ontario, Department of Economics.
  5. V. Joseph Hotz & Guido W. Imbens & Jacob A. Klerman, 2006. "Evaluating the Differential Effects of Alternative Welfare-to-Work Training Components: A Reanalysis of the California GAIN Program," Journal of Labor Economics, University of Chicago Press, University of Chicago Press, vol. 24(3), pages 521-566, July.
  6. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
  7. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," NBER Working Papers 6699, National Bureau of Economic Research, Inc.
  8. Rajeev H. Dehejia & Sadek Wahba, 1998. "Causal Effects in Non-Experimental Studies: Re-Evaluating the Evaluation of Training Programs," NBER Working Papers 6586, National Bureau of Economic Research, Inc.
  9. Dyke, Andrew & Heinrich, Carolyn J. & Mueser, Peter R. & Troske, Kenneth, 2005. "The Effects of Welfare-to-Work Program Activities on Labor Market Outcomes," IZA Discussion Papers 1520, Institute for the Study of Labor (IZA).
  10. Rajeev H. Dehejia & Sadek Wahba, 2002. "Propensity score matching methods for non-experimental causal studies," Discussion Papers, Columbia University, Department of Economics 0102-14, Columbia University, Department of Economics.
  11. Orley Ashenfelter & David Card, 1984. "Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs," Working Papers, Princeton University, Department of Economics, Industrial Relations Section. 554, Princeton University, Department of Economics, Industrial Relations Section..
  12. Imbens, Guido & Abadie, Alberto, 2008. "On the Failure of the Bootstrap for Matching Estimators," Scholarly Articles 3043415, Harvard University Department of Economics.
  13. Dan A. Black & Jeffrey Smith, 2003. "How Robust is the Evidence on the Effects of College Quality? Evidence From Matching," University of Western Ontario, CIBC Centre for Human Capital and Productivity Working Papers, University of Western Ontario, CIBC Centre for Human Capital and Productivity 20033, University of Western Ontario, CIBC Centre for Human Capital and Productivity.
  14. Ben B. Hansen, 2004. "Full Matching in an Observational Study of Coaching for the SAT," Journal of the American Statistical Association, American Statistical Association, American Statistical Association, vol. 99, pages 609-618, January.
  15. Peter Hall & Jeff Racine & Qi Li, 2004. "Cross-Validation and the Estimation of Conditional Probability Densities," Journal of the American Statistical Association, American Statistical Association, American Statistical Association, vol. 99, pages 1015-1026, December.
  16. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2000. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," NBER Technical Working Papers, National Bureau of Economic Research, Inc 0251, National Bureau of Economic Research, Inc.
  17. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, Elsevier, vol. 125(1-2), pages 305-353.
  18. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, American Economic Association, vol. 76(4), pages 604-20, September.
  19. Lorraine Dearden & Javier Ferri & Costas Meghir, 2002. "The Effect Of School Quality On Educational Attainment And Wages," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 1-20, February.
  20. Joshua D. Angrist & Jinyong Hahn, 1999. "When to Control for Covariates? Panel-Asymptotic Results for Estimates of Treatment Effects," NBER Technical Working Papers, National Bureau of Economic Research, Inc 0241, National Bureau of Economic Research, Inc.
  21. Bassi, Laurie J, 1984. "Estimating the Effect of Training Programs with Non-Random Selection," The Review of Economics and Statistics, MIT Press, vol. 66(1), pages 36-43, February.
  22. Burt S. Barnow, 1987. "The Impact of CETA Programs on Earnings: A Review of the Literature," Journal of Human Resources, University of Wisconsin Press, vol. 22(2), pages 157-193.
  23. Heckman, J.J. & Hotz, V.J., 1988. "Choosing Among Alternative Nonexperimental Methods For Estimating The Impact Of Social Programs: The Case Of Manpower Training," University of Chicago - Economics Research Center, Chicago - Economics Research Center 88-12, Chicago - Economics Research Center.
  24. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February.
  25. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, Elsevier, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097 Elsevier.
  26. Alberto Abadie & Guido W. Imbens, 2006. "Large Sample Properties of Matching Estimators for Average Treatment Effects," Econometrica, Econometric Society, Econometric Society, vol. 74(1), pages 235-267, 01.
  27. Markus Frölich, 2004. "Finite-Sample Properties of Propensity-Score Matching and Weighting Estimators," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 77-90, February.
  28. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, American Economic Association, vol. 9(2), pages 85-110, Spring.
  29. Heckman, James J & Ichimura, Hidehiko & Todd, Petra E, 1997. "Matching as an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," Review of Economic Studies, Wiley Blackwell, Wiley Blackwell, vol. 64(4), pages 605-54, October.
  30. Zhong Zhao, 2004. "Using Matching to Estimate Treatment Effects: Data Requirements, Matching Metrics, and Monte Carlo Evidence," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 91-107, February.
  31. Heckman, James J & Smith, Jeffrey A, 1999. "The Pre-programme Earnings Dip and the Determinants of Participation in a Social Programme. Implications for Simple Programme Evaluation Strategies," Economic Journal, Royal Economic Society, Royal Economic Society, vol. 109(457), pages 313-48, July.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
This item has more than 25 citations. To prevent cluttering this page, these citations are listed on a separate page.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp786. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Mark Fallak).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.