This file is part of IDEAS, which uses RePEc data


[ Papers | Articles | Software | Books | Chapters | Authors | Institutions | JEL Classification | NEP reports | Search | New papers by email | Author registration | Rankings | Volunteers | FAQ | Blog | Help! ]

Using State Administrative Data to Measure Program Performance

Author info | Abstract | Publisher info | Download info | Related research | Statistics
Author Info
Peter R. Mueser () (Department of Economics, University of Missouri-Columbia)
Kenneth Troske
Alexey Gorislavsky

Additional information is available for the following registered author(s):

Abstract

This paper uses administrative data from Missouri to examine the sensitivity of job training program impact estimates based on alternative nonexperimental methods. In addition to simple regression adjustment, we consider Mahalanobis distance matching and a variety of methods using propensity score matching. In each case, we consider estimates based on levels of post-program earnings as well as difference-in-difference estimates based on comparison of pre and post-program earnings. Specification tests suggest that the difference-in-difference estimator may provide a better measure of program impact. We find that propensity score matching is generally most effective, but the detailed implementation of the method is not of critical importance. Our analyses demonstrate that existing data available at the state level can be used to obtain useful estimates of program impact.

Download Info
To download:

If you experience problems downloading a file, check if you have the proper application to view it first. Information about this may be contained in the File-Format links below. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://economics.missouri.edu/working-papers/2003/WP0309_mueser.pdf
File Format: application/pdf
File Function:
Download Restriction: no

Publisher Info
Paper provided by Department of Economics, University of Missouri in its series Working Papers with number 0309.

Download reference. The following formats are available: HTML (with abstract), plain text (with abstract), BibTeX, RIS (EndNote, RefMan, ProCite), ReDIF
Length: 15 pgs.
Date of creation: 20 May 2003
Date of revision:
Handle: RePEc:umc:wpaper:0309

Note: Paper updated as WP 05-20
Contact details of provider:
Postal: 118 Professional Building, Columbia, MO 65211
Phone: (573) 882-0063
Fax: (573) 882-2697
Web page: http://economics.missouri.edu/
More information through EDIRC

For technical questions regarding this item, or to correct its listing, contact: (Mark Stratton).

Related research
Keywords: Noexperimental Methods; Matching; Difference-in-Difference;

Other versions of this item:

Find related papers by JEL classification:
H43 - Public Economics - - Publicly Provided Goods - - - Project Evaluation; Social Discount Rate
I38 - Health, Education, and Welfare - - Welfare and Poverty - - - Government Programs; Provision and Effects of Welfare Programs
I28 - Health, Education, and Welfare - - Education - - - Government Policy

This paper has been announced in the following NEP Reports:

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
  1. Heckman, James J & Smith, Jeffrey A, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring. [Downloadable!] (restricted)
  2. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," Econometrica, Econometric Society, vol. 66(5), pages 1017-1098, September.
    Other versions:
  3. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-20, September. [Downloadable!] (restricted)
  4. Rajeev H. Dehejia & Sadek Wahba, 2002. "Propensity Score-Matching Methods For Nonexperimental Causal Studies," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 151-161, February. [Downloadable!] (restricted)
    Other versions:
  5. Ben B. Hansen, 2004. "Full Matching in an Observational Study of Coaching for the SAT," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 609-618, January. [Downloadable!] (restricted)
  6. Peter R. Mueser & Kyung-Seong Jeon & Andrew Dyke & Carolyn J. Heinrich & Kenneth R. Troske, 2006. "The Effects of Welfare-to-Work Program Activities on Labor Market Outcomes," Working Papers 0602, Department of Economics, University of Missouri. [Downloadable!]
    Other versions:
  7. Peter Hall & Jeff Racine & Qi Li, 2004. "Cross-Validation and the Estimation of Conditional Probability Densities," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 1015-1026, December. [Downloadable!] (restricted)
  8. David Card & Daniel Sullivan, 1986. "Measuring the Effect of CETA Participation on Movements In and Out of Employment," Working Papers 586, Princeton University, Department of Economics, Industrial Relations Section.. [Downloadable!]
  9. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, 07. [Downloadable!] (restricted)
    Other versions:
  10. Zhong Zhao, 2004. "Using Matching to Estimate Treatment Effects: Data Requirements, Matching Metrics, and Monte Carlo Evidence," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 91-107, 08. [Downloadable!] (restricted)
  11. Heckman, James J & Ichimura, Hidehiko & Todd, Petra, 1998. "Matching as an Econometric Evaluation Estimator," Review of Economic Studies, Blackwell Publishing, vol. 65(2), pages 261-94, April. [Downloadable!] (restricted)
  12. Alberto Abadie & Guido W. Imbens, 2006. "On the Failure of the Bootstrap for Matching Estimators," NBER Technical Working Papers 0325, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
    Other versions:
  13. Heckman, James J & Ichimura, Hidehiko & Todd, Petra E, 1997. "Matching as an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," Review of Economic Studies, Blackwell Publishing, vol. 64(4), pages 605-54, October. [Downloadable!] (restricted)
  14. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353. [Downloadable!] (restricted)
    Other versions:
  15. James Heckman & Neil Hohmann & Jeffrey Smith & Michael Khoo, 2000. "Substitution And Dropout Bias In Social Experiments: A Study Of An Influential Social Experiment," The Quarterly Journal of Economics, MIT Press, vol. 115(2), pages 651-694, May. [Downloadable!] (restricted)
    Other versions:
  16. Rajeev H. Dehejia & Sadek Wahba, 1998. "Causal Effects in Non-Experimental Studies: Re-Evaluating the Evaluation of Training Programs," NBER Working Papers 6586, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
  17. Alberto Abadie & Guido W. Imbens, 2006. "Large Sample Properties of Matching Estimators for Average Treatment Effects," Econometrica, Econometric Society, vol. 74(1), pages 235-267, 01. [Downloadable!] (restricted)
  18. James J. Heckman, 1989. "Choosing Among Alternative Nonexperimental Methods for Estimating the Impact of Social Programs: The Case of Manpower Training," NBER Working Papers 2861, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
    Other versions:
  19. V. Joseph Hotz & Guido W. Imbens & Jacob A. Klerman, 2006. "Evaluating the Differential Effects of Alternative Welfare-to-Work Training Components: A Reanalysis of the California GAIN Program," Journal of Labor Economics, University of Chicago Press, vol. 24(3), pages 521-566, July. [Downloadable!]
  20. Heckman, James J & Smith, Jeffrey A, 1999. "The Pre-programme Earnings Dip and the Determinants of Participation in a Social Programme. Implications for Simple Programme Evaluation Strategies," Economic Journal, Royal Economic Society, vol. 109(457), pages 313-48, July. [Downloadable!] (restricted)
  21. Black, Dan A. & Smith, J.A.Jeffrey A., 2004. "How robust is the evidence on the effects of college quality? Evidence from matching," Journal of Econometrics, Elsevier, vol. 121(1-2), pages 99-124. [Downloadable!] (restricted)
    Other versions:
  22. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097 Elsevier. [Downloadable!] (restricted)
  23. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February. [Downloadable!] (restricted)
  24. Joshua D. Angrist & Jinyong Hahn, 1999. "When to Control for Covariates? Panel-Asymptotic Results for Estimates of Treatment Effects," NBER Technical Working Papers 0241, National Bureau of Economic Research, Inc. [Downloadable!] (restricted)
  25. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, 06. [Downloadable!] (restricted)
    Other versions:
  26. Lorraine Dearden & Javier Ferri & Costas Meghir, 2002. "The Effect Of School Quality On Educational Attainment And Wages," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 1-20, February. [Downloadable!] (restricted)
    Other versions:
  27. Ashenfelter, Orley & Card, David, 1985. "Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs," The Review of Economics and Statistics, MIT Press, vol. 67(4), pages 648-60, November. [Downloadable!] (restricted)
    Other versions:
  28. Markus Frölich, 2004. "Finite-Sample Properties of Propensity-Score Matching and Weighting Estimators," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 77-90, 07. [Downloadable!] (restricted)
  29. Friedlander, Daniel & Robins, Philip K, 1995. "Evaluating Program Evaluations: New Evidence on Commonly Used Nonexperimental Methods," American Economic Review, American Economic Association, vol. 85(4), pages 923-37, September. [Downloadable!] (restricted)
Full references

Cited by:
(explanations, Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.)

  1. Peter R. Mueser & Kyung-Seong Jeon & Andrew Dyke & Carolyn J. Heinrich & Kenneth R. Troske, 2006. "The Effects of Welfare-to-Work Program Activities on Labor Market Outcomes," Working Papers 0602, Department of Economics, University of Missouri. [Downloadable!]
    Other versions:
  2. Lechner, Michael & Vazquez-Alvarez, Rosalia, 2004. "The Effect of Disability on Labour Market Outcomes in Germany: Evidence from Matching," CEPR Discussion Papers 4223, C.E.P.R. Discussion Papers. [Downloadable!] (restricted)
    Other versions:
  3. Zhong Zhao, 2005. "Sensitivity of Propensity Score Methods to the Specifications," IZA Discussion Papers 1873, Institute for the Study of Labor (IZA). [Downloadable!]
    Other versions:
  4. Rachel Gordon & Carolyn Heinrich, 2009. "The potential of a couples approach to employment assistance: results of a nonexperimental evaluation," Review of Economics of the Household, Springer, vol. 7(2), pages 133-158, June. [Downloadable!] (restricted)
  5. Kevin Hollenbeck, 2008. "Sensitivity Testing of Net Impact Estimates of Workforce Development Programs Using Administrative Data," Staff Working Papers 08-139, W.E. Upjohn Institute for Employment Research. [Downloadable!] (restricted)
  6. Kevin Hollenbeck, 2004. "Some Reflections on the Use of Administrative Data to Estimate the Net Impacts of Workforce Programs in Washington State," Staff Working Papers 04-109, W.E. Upjohn Institute for Employment Research. [Downloadable!] (restricted)
  7. Ulf Rinne & Marc Schneider & Arne Uhlendorff, 2007. "Too Bad to Benefit?: Effect Heterogeneity of Public Training Programs," Discussion Papers of DIW Berlin 749, DIW Berlin, German Institute for Economic Research. [Downloadable!]
    Other versions:
  8. Carolyn Heinrich & Marcelo Cabrol, 2005. "An Impact Evaluation of the National Student Scholarship Program in Argentina," OVE Working Papers 0605, Inter-American Development Bank, Office of Evaluation and Oversight (OVE). [Downloadable!]
  9. Naci Mocan & Erdal Tekin, 2006. "Catholic Schools and Bad Behavior: A Propensity Score Matching Analysis," Contributions to Economic Analysis & Policy, Berkeley Electronic Press, vol. 5(1), pages 1403-1403. [Downloadable!] (restricted)
    Other versions:
  10. Alfonso Flores-Lagunes & Arturo Gonzalez & Todd Neumann, 2005. "Learning but Not Earning? The Value of Job Corps Training for Hispanic Youths," IZA Discussion Papers 1638, Institute for the Study of Labor (IZA). [Downloadable!]
Statistics
Access and download statistics

Did you know? Citation analysis on IDEAS includes online papers that are freely accessible and whose text could be automatically analyzed, currently about 210000 papers.

This page was last updated on 2009-11-3.

This information is provided to you by IDEAS at the Department of Economics, College of Liberal Arts and Sciences, University of Connecticut using RePEc data on a server sponsored by the Society for Economic Dynamics.