IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp4569.html
   My bibliography  Save this paper

New Estimates of Public Employment and Training Program Net Impacts: A Nonexperimental Evaluation of the Workforce Investment Act Program

Author

Listed:
  • Heinrich, Carolyn J.

    (University of Texas at Austin)

  • Mueser, Peter R.

    (University of Missouri, Columbia)

  • Troske, Kenneth

    (University of Kentucky)

  • Jeon, Kyung-Seong

    (University of Missouri-Columbia)

  • Kahvecioglu, Daver C.

    (Impaq International)

Abstract

This paper presents nonexperimental net impact estimates for the Adult and Dislocated Worker programs under the Workforce Investment Act (WIA), the primary federal job training program in the U.S, based on administrative data from 12 states, covering approximately 160,000 WIA participants and nearly 3 million comparison group members. The key measure of interest is the difference in average quarterly earnings or employment attributable to WIA program participation for those who participate, estimated for up to four years following entry into the program using propensity score matching methods. The results for the average participant in the WIA Adult program show that participating is associated with a several-hundred-dollar increase in quarterly earnings. Adult program participants who obtain training have lower earnings in the months during training and the year after exit than those who don’t receive training, but they catch up within 10 quarters, ultimately registering large total gains. The marginal benefits of training exceed, on average, $400 in earnings each quarter three years after program entry. Dislocated Workers experience several quarters for which earnings are depressed relative to comparison group workers after entering WIA, and although their earnings ultimately match or overtake the comparison group, the benefits they obtain are smaller than for those in the Adult program.

Suggested Citation

  • Heinrich, Carolyn J. & Mueser, Peter R. & Troske, Kenneth & Jeon, Kyung-Seong & Kahvecioglu, Daver C., 2009. "New Estimates of Public Employment and Training Program Net Impacts: A Nonexperimental Evaluation of the Workforce Investment Act Program," IZA Discussion Papers 4569, Institute of Labor Economics (IZA).
  • Handle: RePEc:iza:izadps:dp4569
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp4569.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. James J. Heckman & Petra E. Todd, 2009. "A note on adapting propensity score matching and selection models to choice based samples," Econometrics Journal, Royal Economic Society, vol. 12(s1), pages 230-234, January.
    2. Steven Glazerman & Dan M. Levy & David Myers, 2003. "Nonexperimental Versus Experimental Estimates of Earnings Impacts," The ANNALS of the American Academy of Political and Social Science, , vol. 589(1), pages 63-93, September.
    3. V. Joseph Hotz & Guido W. Imbens & Jacob A. Klerman, 2006. "Evaluating the Differential Effects of Alternative Welfare-to-Work Training Components: A Reanalysis of the California GAIN Program," Journal of Labor Economics, University of Chicago Press, vol. 24(3), pages 521-566, July.
    4. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February.
    5. Michael Lechner & Conny Wunsch, 2009. "Are Training Programs More Effective When Unemployment Is High?," Journal of Labor Economics, University of Chicago Press, vol. 27(4), pages 653-692, October.
    6. David Card & Jochen Kluve & Andrea Weber, 2010. "Active Labour Market Policy Evaluations: A Meta-Analysis," Economic Journal, Royal Economic Society, vol. 120(548), pages 452-477, November.
    7. Marco Caliendo & Sabine Kopeinig, 2008. "Some Practical Guidance For The Implementation Of Propensity Score Matching," Journal of Economic Surveys, Wiley Blackwell, vol. 22(1), pages 31-72, February.
    8. Fredriksson, Peter & Johansson, Per, 2008. "Dynamic Treatment Assignment," Journal of Business & Economic Statistics, American Statistical Association, vol. 26, pages 435-445.
    9. Heinrich, Carolyn J., 2007. "Demand and Supply-Side Determinants of Conditional Cash Transfer Program Effectiveness," World Development, Elsevier, vol. 35(1), pages 121-143, January.
    10. Barbara Sianesi, 2004. "An Evaluation of the Swedish System of Active Labor Market Programs in the 1990s," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 133-155, February.
    11. Andrew Dyke & Carolyn J. Heinrich & Peter R. Mueser & Kenneth R. Troske & Kyung-Seong Jeon, 2006. "The Effects of Welfare-to-Work Program Activities on Labor Market Outcomes," Journal of Labor Economics, University of Chicago Press, vol. 24(3), pages 567-608, July.
    12. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    13. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    14. Dan A. Black & Jeffrey A. Smith & Mark C. Berger & Brett J. Noel, 2003. "Is the Threat of Reemployment Services More Effective Than the Services Themselves? Evidence from Random Assignment in the UI System," American Economic Review, American Economic Association, vol. 93(4), pages 1313-1327, September.
    15. Heckman, James J & Smith, Jeffrey A, 1999. "The Pre-programme Earnings Dip and the Determinants of Participation in a Social Programme. Implications for Simple Programme Evaluation Strategies," Economic Journal, Royal Economic Society, vol. 109(457), pages 313-348, July.
    16. Alberto Abadie & Guido W. Imbens, 2008. "On the Failure of the Bootstrap for Matching Estimators," Econometrica, Econometric Society, vol. 76(6), pages 1537-1557, November.
    17. Smith, Jeffrey & Todd, Petra, 2005. "Rejoinder," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 365-375.
    18. Peter R. Mueser & Kenneth R. Troske & Alexey Gorislavsky, 2007. "Using State Administrative Data to Measure Program Performance," The Review of Economics and Statistics, MIT Press, vol. 89(4), pages 761-783, November.
    19. Peikes, Deborah N. & Moreno, Lorenzo & Orzol, Sean Michael, 2008. "Propensity Score Matching: A Note of Caution for Evaluators of Social Programs," The American Statistician, American Statistical Association, vol. 62, pages 222-231, August.
    20. Rajeev H. Dehejia & Sadek Wahba, 2002. "Propensity Score-Matching Methods For Nonexperimental Causal Studies," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 151-161, February.
    21. repec:adr:anecst:y:2008:i:91-92:p:10 is not listed on IDEAS
    22. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097, Elsevier.
    23. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
    24. repec:mpr:mprres:3694 is not listed on IDEAS
    25. Christopher J. O'Leary, 2004. "Evaluating the Effectiveness of Labor Exchange Services," Book chapters authored by Upjohn Institute researchers, in: David E. Balducchi & Randall W. Eberts & Christopher J. O'Leary (ed.), Labor Exchange Policy in the United States, chapter 5, pages 135-178, W.E. Upjohn Institute for Employment Research.
    26. Deborah N. Peikes & Lorenzo Moreno & Sean Michael Orzol, "undated". "Propensity Score Matching: A Note of Caution for Evaluators of Social Programs," Mathematica Policy Research Reports dd0866e4646a4e0ea77079d5b, Mathematica Policy Research.
    27. Jose C. Galdo & Jeffrey Smith & Dan Black, 2008. "Bandwidth Selection and the Estimation of Treatment Effects with Unbalanced Data," Annals of Economics and Statistics, GENES, issue 91-92, pages 189-216.
    28. Alberto Abadie & Guido W. Imbens, 2006. "Large Sample Properties of Matching Estimators for Average Treatment Effects," Econometrica, Econometric Society, vol. 74(1), pages 235-267, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Dolton, Peter & Smith, Jeffrey A., 2011. "The Impact of the UK New Deal for Lone Parents on Benefit Receipt," IZA Discussion Papers 5491, Institute of Labor Economics (IZA).
    2. Martin Huber & Michael Lechner & Anthony Strittmatter, 2018. "Direct and indirect effects of training vouchers for the unemployed," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(2), pages 441-463, February.
    3. Holzer, Harry J., 2012. "Raising Job Quality and Worker Skills in the US: Creating More Effective Education and Workforce Development Systems in States," IZA Policy Papers 42, Institute of Labor Economics (IZA).
    4. Martin Biewen & Bernd Fitzenberger & Aderonke Osikominu & Marie Paul, 2014. "The Effectiveness of Public-Sponsored Training Revisited: The Importance of Data and Methodological Choices," Journal of Labor Economics, University of Chicago Press, vol. 32(4), pages 837-897.
    5. Gueorgui Kambourov & Iourii Manovskii & Miana Plesca, 2020. "Occupational mobility and the returns to training," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 53(1), pages 174-211, February.
    6. Andrew Barr & Sarah Turner, 2018. "A Letter and Encouragement: Does Information Increase Postsecondary Enrollment of UI Recipients?," American Economic Journal: Economic Policy, American Economic Association, vol. 10(3), pages 42-68, August.
    7. Barr, Andrew & Turner, Sarah, 2015. "Out of work and into school: Labor market policies and college enrollment during the Great Recession," Journal of Public Economics, Elsevier, vol. 124(C), pages 63-73.
    8. Singer, Christine & Toomet, Ott-Siim, 2013. "On government-subsidized training programs for older workers," IAB-Discussion Paper 201321, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    9. Lechner, Michael & Wunsch, Conny, 2013. "Sensitivity of matching-based program evaluations to the availability of control variables," Labour Economics, Elsevier, vol. 21(C), pages 111-121.
    10. Bernd Fitzenberger & Olga Orlanski & Aderonke Osikominu & Marie Paul, 2013. "Déjà Vu? Short-term training in Germany 1980–1992 and 2000–2003," Empirical Economics, Springer, vol. 44(1), pages 289-328, February.
    11. Jonah Deutsch & Katherine Allison-Clark & Armando Yañez, "undated". "A Research Evidence Scan of Key Strategies Related to WIOA," Mathematica Policy Research Reports dd605a2bfed047adb97db7c7c, Mathematica Policy Research.
    12. Fredrik Andersson & Harry J. Holzer & Julia I. Lane & David Rosenblum & Jeffrey Smith, 2024. "Does Federally Funded Job Training Work? Nonexperimental Estimates of WIA Training Impacts Using Longitudinal Data on Workers and Firms," Journal of Human Resources, University of Wisconsin Press, vol. 59(4), pages 1244-1283.
    13. Adrian Fleissig, 2014. "Return on Investment from Training Programs and Intensive Services," Atlantic Economic Journal, Springer;International Atlantic Economic Society, vol. 42(1), pages 39-51, March.
    14. Jones, Stephen, 2012. "The Effectiveness of Training for Displaced Workers with Long Prior Job Tenure," CLSSRN working papers clsrn_admin-2012-3, Vancouver School of Economics, revised 29 Jan 2012.
    15. Jones, Michael, 2017. "The Effect of Job Readiness Programs on Criminal Behavior," MPRA Paper 81908, University Library of Munich, Germany.
    16. Schwerdt, Guido & Messer, Dolores & Woessmann, Ludger & Wolter, Stefan C., 2012. "The impact of an adult education voucher program: Evidence from a randomized field experiment," Journal of Public Economics, Elsevier, vol. 96(7-8), pages 569-583.
    17. Doerr, Annabelle & Strittmatter, Anthony, 2014. "Assignment Mechanisms, Selection Criteria, and the Effectiveness of Training Programs," Economics Working Paper Series 1421, University of St. Gallen, School of Economics and Political Science, revised May 2017.
    18. Paul, Marie & Dörr, Annabelle & Fitzenberger, Bernd & Kruppe, Thomas & Strittmatter, Anthony, 2013. "The Award of a Training Voucher and Labor Market Outcomes," VfS Annual Conference 2013 (Duesseldorf): Competition Policy and Regulation in a Global Economic Order 79918, Verein für Socialpolitik / German Economic Association.
    19. Harry Holzer, 2012. "Good workers for good jobs: improving education and workforce systems in the US," IZA Journal of Labor Policy, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 1(1), pages 1-19, December.
    20. Dolores Messer & Guido Schwerdt & Ludger Woessmann & Stefan C. Wolter, 2013. "Labor Market Effects of Adult Education Vouchers: Evidence from a Randomized Field Experiment," Economics of Education Working Paper Series 0094, University of Zurich, Department of Business Administration (IBW).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lechner, Michael & Wunsch, Conny, 2013. "Sensitivity of matching-based program evaluations to the availability of control variables," Labour Economics, Elsevier, vol. 21(C), pages 111-121.
    2. Caliendo, Marco & Mahlstedt, Robert & Mitnik, Oscar A., 2017. "Unobservable, but unimportant? The relevance of usually unobserved variables for the evaluation of labor market policies," Labour Economics, Elsevier, vol. 46(C), pages 14-25.
    3. Peter R. Mueser & Kenneth R. Troske & Alexey Gorislavsky, 2007. "Using State Administrative Data to Measure Program Performance," The Review of Economics and Statistics, MIT Press, vol. 89(4), pages 761-783, November.
    4. Gueorgui Kambourov & Iourii Manovskii & Miana Plesca, 2020. "Occupational mobility and the returns to training," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 53(1), pages 174-211, February.
    5. Martin Biewen & Bernd Fitzenberger & Aderonke Osikominu & Marie Paul, 2014. "The Effectiveness of Public-Sponsored Training Revisited: The Importance of Data and Methodological Choices," Journal of Labor Economics, University of Chicago Press, vol. 32(4), pages 837-897.
    6. Seonho Shin, 2022. "Evaluating the Effect of the Matching Grant Program for Refugees: An Observational Study Using Matching, Weighting, and the Mantel-Haenszel Test," Journal of Labor Research, Springer, vol. 43(1), pages 103-133, March.
    7. Huber, Martin & Lechner, Michael & Wunsch, Conny, 2013. "The performance of estimators based on the propensity score," Journal of Econometrics, Elsevier, vol. 175(1), pages 1-21.
    8. Dolton, Peter & Smith, Jeffrey A., 2011. "The Impact of the UK New Deal for Lone Parents on Benefit Receipt," IZA Discussion Papers 5491, Institute of Labor Economics (IZA).
    9. Jones A.M & Rice N, 2009. "Econometric Evaluation of Health Policies," Health, Econometrics and Data Group (HEDG) Working Papers 09/09, HEDG, c/o Department of Economics, University of York.
    10. Carlos A. Flores & Oscar A. Mitnik, 2009. "Evaluating Nonexperimental Estimators for Multiple Treatments: Evidence from Experimental Data," Working Papers 2010-10, University of Miami, Department of Economics.
    11. Hagen, Tobias, 2016. "Econometric Evaluation of a Placement Coaching Program for Recipients of Disability Insurance Benefits in Switzerland," VfS Annual Conference 2016 (Augsburg): Demographic Change 145736, Verein für Socialpolitik / German Economic Association.
    12. Ulf Rinne & Arne Uhlendorff & Zhong Zhao, 2013. "Vouchers and caseworkers in training programs for the unemployed," Empirical Economics, Springer, vol. 45(3), pages 1089-1127, December.
    13. Hagen, Tobias, 2016. "Econometric evaluation of a placement coaching program for recipients of disability insurance benefits in Switzerland," Working Paper Series 10, Frankfurt University of Applied Sciences, Faculty of Business and Law.
    14. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    15. Kluve, Jochen & Lehmann, Hartmut & Schmidt, Christoph M., 2008. "Disentangling Treatment Effects of Active Labor Market Policies: The Role of Labor Force Status Sequences," Labour Economics, Elsevier, vol. 15(6), pages 1270-1295, December.
    16. Muller, Paul & van der Klaauw, Bas & Heyma, Arjan, 2017. "Comparing Econometric Methods to Empirically Evaluate Job-Search Assistance," IZA Discussion Papers 10531, Institute of Labor Economics (IZA).
    17. Martin Huber & Michael Lechner & Andreas Steinmayr, 2015. "Radius matching on the propensity score with bias adjustment: tuning parameters and finite sample behaviour," Empirical Economics, Springer, vol. 49(1), pages 1-31, August.
    18. Steven Lehrer & Gregory Kordas, 2013. "Matching using semiparametric propensity scores," Empirical Economics, Springer, vol. 44(1), pages 13-45, February.
    19. Bergemann, Annette & van den Berg, Gerard J., 2008. "From Giving Birth to Paid Labor: The Effects of Adult Education for Prime-Aged Mothers," IZA Discussion Papers 3600, Institute of Labor Economics (IZA).
    20. Fortson, Kenneth & Gleason, Philip & Kopa, Emma & Verbitsky-Savitz, Natalya, 2015. "Horseshoes, hand grenades, and treatment effects? Reassessing whether nonexperimental estimators are biased," Economics of Education Review, Elsevier, vol. 44(C), pages 100-113.

    More about this item

    Keywords

    nonexperimental program evaluation; workforce investment act;

    JEL classification:

    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity
    • J48 - Labor and Demographic Economics - - Particular Labor Markets - - - Particular Labor Markets; Public Policy

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp4569. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.