This file is part of IDEAS, which uses RePEc data


[ Papers | Articles | Software | Books | Chapters | Authors | Institutions | JEL Classification | NEP reports | Search | New papers by email | Author registration | Rankings | Volunteers | FAQ | Blog | Help! ]

Using State Administrative Data to Measure Program Performance

Author info | Abstract | Publisher info | Download info | Related research | Statistics
Author Info
Peter R. Mueser (University of Missouri�Columbia and IZA)
Kenneth R. Troske (University of Kentucky�Lexington and IZA)
Alexey Gorislavsky (University of Missouri�Columbia)

Additional information is available for the following registered author(s):

Abstract

We use administrative data from Missouri to examine the sensitivity of earnings impact estimates for a job training program based on alternative nonexperimental methods. We consider regression adjustment, Mahalanobis distance matching, and various methods using propensity-score matching, examining both cross-sectional estimates and difference-in-difference estimates. Specification tests suggest that the difference-in-difference estimator may provide a better measure of program impact. We find that propensity-score matching is most effective, but the detailed implementation is not of critical importance. Our analyses demonstrate that existing data can be used to obtain useful estimates of program impact. Copyright by the President and Fellows of Harvard College and the Massachusetts Institute of Technology.

Download Info
To download:

If you experience problems downloading a file, check if you have the proper application to view it first. Information about this may be contained in the File-Format links below. In case of further problems read the IDEAS help file. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.mitpressjournals.org/doi/pdfplus/10.1162/rest.89.4.761
File Format: text/html
File Function: link to full text
Download Restriction: Access to full text is restricted to subscribers.

As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

Publisher Info
Article provided by MIT Press in its journal The Review of Economics and Statistics.

Volume (Year): 89 (2007)
Issue (Month): 4 (09)
Pages: 761-783
Download reference. The following formats are available: HTML, plain text, BibTeX, RIS (EndNote), ReDIF
Handle: RePEc:tpr:restat:v:89:y:2007:i:4:p:761-783

Contact details of provider:
Web page: http://mitpress.mit.edu/journals/

Order Information:
Web: http://mitpress.mit.edu/journal-home.tcl?issn=00346535

For technical questions regarding this item, or to correct its listing, contact: (Christopher F. Baum).

Related research
Keywords:

Other versions of this item:

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
  1. Heckman, James J & Smith, Jeffrey A, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring. [Downloadable!] (restricted)
  2. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," Econometrica, Econometric Society, vol. 66(5), pages 1017-1098, September.
    Other versions:
  3. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-20, September. [Downloadable!] (restricted)
  4. Rajeev H. Dehejia & Sadek Wahba, 2002. "Propensity Score-Matching Methods For Nonexperimental Causal Studies," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 151-161, February. [Downloadable!] (restricted)
    Other versions:
  5. Ben B. Hansen, 2004. "Full Matching in an Observational Study of Coaching for the SAT," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 609-618, January. [Downloadable!] (restricted)
  6. Peter R. Mueser & Kyung-Seong Jeon & Andrew Dyke & Carolyn J. Heinrich & Kenneth R. Troske, 2006. "The Effects of Welfare-to-Work Program Activities on Labor Market Outcomes," Working Papers 0602, Department of Economics, University of Missouri. [Downloadable!]
    Other versions:
  7. Peter Hall & Jeff Racine & Qi Li, 2004. "Cross-Validation and the Estimation of Conditional Probability Densities," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 1015-1026, December. [Downloadable!] (restricted)
  8. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, 07. [Downloadable!] (restricted)
    Other versions:
  9. Zhong Zhao, 2004. "Using Matching to Estimate Treatment Effects: Data Requirements, Matching Metrics, and Monte Carlo Evidence," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 91-107, 08. [Downloadable!] (restricted)
  10. Heckman, James J & Ichimura, Hidehiko & Todd, Petra, 1998. "Matching as an Econometric Evaluation Estimator," Review of Economic Studies, Blackwell Publishing, vol. 65(2), pages 261-94, April. [Downloadable!] (restricted)
  11. Alberto Abadie & Guido W. Imbens, 2006. "On the Failure of the Bootstrap for Matching Estimators," NBER Technical Working Papers 0325, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
  12. Heckman, James J & Ichimura, Hidehiko & Todd, Petra E, 1997. "Matching as an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," Review of Economic Studies, Blackwell Publishing, vol. 64(4), pages 605-54, October. [Downloadable!] (restricted)
  13. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353. [Downloadable!] (restricted)
    Other versions:
  14. James Heckman & Neil Hohmann & Jeffrey Smith & Michael Khoo, 2000. "Substitution And Dropout Bias In Social Experiments: A Study Of An Influential Social Experiment," The Quarterly Journal of Economics, MIT Press, vol. 115(2), pages 651-694, May. [Downloadable!] (restricted)
    Other versions:
  15. Rajeev H. Dehejia & Sadek Wahba, 1998. "Causal Effects in Non-Experimental Studies: Re-Evaluating the Evaluation of Training Programs," NBER Working Papers 6586, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
  16. Alberto Abadie & Guido W. Imbens, 2006. "Large Sample Properties of Matching Estimators for Average Treatment Effects," Econometrica, Econometric Society, vol. 74(1), pages 235-267, 01. [Downloadable!] (restricted)
  17. James J. Heckman, 1989. "Choosing Among Alternative Nonexperimental Methods for Estimating the Impact of Social Programs: The Case of Manpower Training," NBER Working Papers 2861, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
    Other versions:
  18. V. Joseph Hotz & Guido W. Imbens & Jacob A. Klerman, 2006. "Evaluating the Differential Effects of Alternative Welfare-to-Work Training Components: A Reanalysis of the California GAIN Program," Journal of Labor Economics, University of Chicago Press, vol. 24(3), pages 521-566, July. [Downloadable!]
  19. Heckman, James J & Smith, Jeffrey A, 1999. "The Pre-programme Earnings Dip and the Determinants of Participation in a Social Programme. Implications for Simple Programme Evaluation Strategies," Economic Journal, Royal Economic Society, vol. 109(457), pages 313-48, July. [Downloadable!] (restricted)
  20. Black, Dan A. & Smith, J.A.Jeffrey A., 2004. "How robust is the evidence on the effects of college quality? Evidence from matching," Journal of Econometrics, Elsevier, vol. 121(1-2), pages 99-124. [Downloadable!] (restricted)
    Other versions:
  21. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097 Elsevier. [Downloadable!] (restricted)
  22. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February. [Downloadable!] (restricted)
  23. Joshua D. Angrist & Jinyong Hahn, 1999. "When to Control for Covariates? Panel-Asymptotic Results for Estimates of Treatment Effects," NBER Technical Working Papers 0241, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
  24. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, 06. [Downloadable!] (restricted)
    Other versions:
  25. Lorraine Dearden & Javier Ferri & Costas Meghir, 2002. "The Effect Of School Quality On Educational Attainment And Wages," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 1-20, February. [Downloadable!] (restricted)
    Other versions:
  26. Ashenfelter, Orley & Card, David, 1985. "Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs," The Review of Economics and Statistics, MIT Press, vol. 67(4), pages 648-60, November. [Downloadable!] (restricted)
    Other versions:
  27. Markus Frölich, 2004. "Finite-Sample Properties of Propensity-Score Matching and Weighting Estimators," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 77-90, 07. [Downloadable!] (restricted)
  28. Friedlander, Daniel & Robins, Philip K, 1995. "Evaluating Program Evaluations: New Evidence on Commonly Used Nonexperimental Methods," American Economic Review, American Economic Association, vol. 85(4), pages 923-37, September. [Downloadable!] (restricted)
Full references

Cited by:
(explanations, Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.)

  1. Lechner, Michael & Vazquez-Alvarez, Rosalia, 2004. "The Effect of Disability on Labour Market Outcomes in Germany: Evidence from Matching," CEPR Discussion Papers 4223, C.E.P.R. Discussion Papers. [Downloadable!] (restricted)
    Other versions:
  2. Kevin Hollenbeck, 2008. "Sensitivity Testing of Net Impact Estimates of Workforce Development Programs Using Administrative Data," Staff Working Papers 08-139, W.E. Upjohn Institute for Employment Research. [Downloadable!] (restricted)
  3. Kevin Hollenbeck, 2004. "Some Reflections on the Use of Administrative Data to Estimate the Net Impacts of Workforce Programs in Washington State," Staff Working Papers 04-109, W.E. Upjohn Institute for Employment Research. [Downloadable!] (restricted)
  4. Carolyn Heinrich & Marcelo Cabrol, 2005. "An Impact Evaluation of the National Student Scholarship Program in Argentina," OVE Working Papers 0605, Inter-American Development Bank, Office of Evaluation and Oversight (OVE). [Downloadable!]
  5. Peter R. Mueser & Kyung-Seong Jeon & Andrew Dyke & Carolyn J. Heinrich & Kenneth R. Troske, 2006. "The Effects of Welfare-to-Work Program Activities on Labor Market Outcomes," Working Papers 0602, Department of Economics, University of Missouri. [Downloadable!]
    Other versions:
  6. Zhong Zhao, 2005. "Sensitivity of Propensity Score Methods to the Specifications," IZA Discussion Papers 1873, Institute for the Study of Labor (IZA). [Downloadable!]
    Other versions:
  7. Ulf Rinne & Marc Schneider & Arne Uhlendorff, 2007. "Too Bad to Benefit? : Effect Heterogeneity of Public Training Programs," Discussion Papers of DIW Berlin 749, DIW Berlin, German Institute for Economic Research. [Downloadable!]
    Other versions:
  8. Alfonso Flores-Lagunes & Arturo Gonzalez & Todd Neumann, 2005. "Learning but Not Earning? The Value of Job Corps Training for Hispanic Youths," IZA Discussion Papers 1638, Institute for the Study of Labor (IZA). [Downloadable!]
Statistics
Access and download statistics

Did you know? You too can volunteer for RePEc, for example by providing information about publications in your institution.

This page was last updated on 2008-8-28.

This information is provided to you by IDEAS at the Department of Economics, College of Liberal Arts and Sciences, University of Connecticut using RePEc data on a server sponsored by the Society for Economic Dynamics.