IDEAS home Printed from https://ideas.repec.org/a/wly/jpamgt/v27y2008i3p556-576.html

Do Job Corps performance measures track program impacts?

Author

Listed:
  • Peter Z. Schochet

    (Mathematica Policy Research, Inc.)

  • John A. Burghardt

    (Mathematica Policy Research, Inc.)

Abstract

Since the 1993 Government Performance and Results Act, performance measurement systems based on short-term program outcomes have been increasingly used to assess the effectiveness of federal programs. This paper examines the association between program performance measures and long-term program impacts, using nine-year follow-up data from a recent large-scale, national experimental evaluation of Job Corps, the nation's largest federal job training program for disadvantaged youths. Job Corps is an important test case because it uses a comprehensive performance system that is widely emulated. We find that impacts on key outcomes are not associated with measured center performance levels. Participants in higherperforming centers had better outcomes; however, the same pattern holds for comparable controls. Thus, the performance measurement system is not achieving the goal of ranking and rewarding centers on the basis of their ability to improve participant outcomes relative to what these outcomes would have been otherwise. © 2008 by the Association for Public Policy Analysis and Management.

Suggested Citation

  • Peter Z. Schochet & John A. Burghardt, 2008. "Do Job Corps performance measures track program impacts?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 27(3), pages 556-576.
    • Handle: RePEc:wly:jpamgt:v:27:y:2008:i:3:p:556-576
    DOI: 10.1002/pam.20356
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1002/pam.20356
    File Function: Link to full text; subscription required
    Download Restriction: no
    File URL: https://libkey.io/10.1002/pam.20356?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. James J. Heckman & Carolyn Heinrich & Jeffrey Smith, 2002. "The Performance of Performance Standards," Journal of Human Resources, University of Wisconsin Press, vol. 37(4), pages 778-811.
    2. Howard S. Bloom, 1984. "Accounting for No-Shows in Experimental Evaluation Designs," Evaluation Review, , vol. 8(2), pages 225-246, April.
    3. Robert S. Gay & Michael E. Borus, 1980. "Validating Performance Indicators for Employment and Training Programs," Journal of Human Resources, University of Wisconsin Press, vol. 15(1), pages 29-48.
    4. Pascal Courty & Gerald Marschke, 2002. "Performance Incentives with Award Constraints," Journal of Human Resources, University of Wisconsin Press, vol. 37(4), pages 812-845.
    5. Peter Z. Schochet & John Burghardt & Steven Glazerman, 2001. "National Job Corps Study: The Impacts of Job Corps on Participants' Employment and Related Outcomes," Mathematica Policy Research Reports db6c4204b8e1408bb0c6289ec, Mathematica Policy Research.
    6. Terry Johnson & Mark Gritz & Russell Jackson & John Burghardt & Carol Boussy & Jan Leonard & Carlyn Orians, 1999. "National Job Corps Study: Report on the Process Analysis," Mathematica Policy Research Reports efc0cd05f0524a049779f797f, Mathematica Policy Research.
    7. Burt S Barnow, 2000. "Exploring the relationship between performance management and program impact: A case study of the job training partnership act," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 19(1), pages 118-141.
    8. repec:mpr:mprres:2951 is not listed on IDEAS
    9. repec:mpr:mprres:2955 is not listed on IDEAS
    10. Sheena McConnell & Steven Glazerman, 2001. "National Job Corps Study: The Benefits and Costs of Job Corps," Mathematica Policy Research Reports 19ff8678a108410587c5dfad0, Mathematica Policy Research.
    11. repec:mpr:mprres:1966 is not listed on IDEAS
    12. Cragg, Michael, 1997. "Performance Incentives in the Public Sector: Evidence from the Job Training Partnership Act," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 13(1), pages 147-168, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rothstein, Jesse & Von Wachter, Till, 2016. "Social Experiments in the Labor Market," Department of Economics, Working Paper Series qt7957p9g6, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
    2. repec:mpr:mprres:7650 is not listed on IDEAS
    3. Burt S. Barnow & Jeffrey Smith, 2015. "Employment and Training Programs," NBER Chapters, in: Economics of Means-Tested Transfer Programs in the United States, Volume 2, pages 127-234, National Bureau of Economic Research, Inc.
    4. Douglas J. Besharov, 2009. "Presidential address: From the Great Society to continuous improvement government: Shifting from “does it work?” to “what would make it better?”," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 28(2), pages 199-220.
    5. Flores-Lagunes, Alfonso & Gonzalez, Arturo & Neumann, Todd C., 2007. "Estimating the Effects of Length of Exposure to a Training Program: The Case of Job Corps," IZA Discussion Papers 2846, IZA Network @ LISER.
    6. Joanne Lee & Peter Z. Schochet & Jillian Berk, "undated". "The External Review of Job Corps: Directions for Future Research," Mathematica Policy Research Reports 376221bbee0d4b40bda431a16, Mathematica Policy Research.
    7. Sameh Hallaq, 2020. "The Palestinian Labor Market over the Last Three Decades," Economics Working Paper Archive wp_976, Levy Economics Institute.
    8. Jane Fortson & Peter Z. Schochet, "undated". "Analysis of Associations Between Contemporaneous Job Corps Performance Measures and Impact Estimates from the National Job Corps Study," Mathematica Policy Research Reports 66ddefe84fb84c1e9a0666e17, Mathematica Policy Research.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Burt S. Barnow & Jeffrey Smith, 2015. "Employment and Training Programs," NBER Chapters, in: Economics of Means-Tested Transfer Programs in the United States, Volume 2, pages 127-234, National Bureau of Economic Research, Inc.
    2. Robert J. LaLonde, 2003. "Employment and Training Programs," NBER Chapters, in: Means-Tested Transfer Programs in the United States, pages 517-586, National Bureau of Economic Research, Inc.
    3. Pascal Courty & Gerald Marschke, 2003. "Making Government Accountable: Lessons from a Federal Job Training Program," The Centre for Market and Public Organisation 03/083, The Centre for Market and Public Organisation, University of Bristol, UK.
    4. Peter Z. Schochet & John Burghardt, 2007. "Using Propensity Scoring to Estimate Program-Related Subgroup Impacts in Experimental Program Evaluations," Evaluation Review, , vol. 31(2), pages 95-120, April.
    5. Courty, Pascal & Marschke, Gerald, 2004. "A General Test of Gaming," CEPR Discussion Papers 4514, C.E.P.R. Discussion Papers.
    6. Peter Z. Schochet, 2021. "Long‐Run Labor Market Effects of the Job Corps Program: Evidence from a Nationally Representative Experiment," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 40(1), pages 128-157, January.
    7. Flores-Lagunes, Alfonso & Gonzalez, Arturo & Neumann, Todd C., 2005. "Learning but Not Earning? The Value of Job Corps Training for Hispanic Youths," IZA Discussion Papers 1638, IZA Network @ LISER.
    8. Peter Z. Schochet & Ronald D'Amico & Jillian Berk & Sarah Dolfin & Nathan Wozny, "undated". "Estimated Impacts for Participants in the Trade Adjustment Assistance (TAA) Program Under the 2002 Amendments," Mathematica Policy Research Reports 582d8723f6884d4eb7a3f95a4, Mathematica Policy Research.
    9. Courty, Pascal & Kim, Do Han & Marschke, Gerald, 2011. "Curbing cream-skimming: Evidence on enrolment incentives," Labour Economics, Elsevier, vol. 18(5), pages 643-655, October.
    10. Carolyn J. Heinrich & Laurence E. Lynn Jr., 1999. "Governance And Performance: The Influence of Program Structure and Management on Job Training Partnership Act (JTPA) Program Outcomes," JCPR Working Papers 107, Northwestern University/University of Chicago Joint Center for Poverty Research.
    11. Annalisa Mastri & Emily Evans & Stacy Pancratz & Carrie Wolfson, "undated". "Employment Research in Brief: An Annotated Bibliography of ETA-Sponsored Studies," Mathematica Policy Research Reports 2f5be979ffa14d3996aa77f33, Mathematica Policy Research.
    12. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097, Elsevier.
    13. Peter Z. Schochet & John Burghardt & Steven Glazerman, 2001. "National Job Corps Study: The Impacts of Job Corps on Participants' Employment and Related Outcomes," Mathematica Policy Research Reports db6c4204b8e1408bb0c6289ec, Mathematica Policy Research.
    14. S. Avdasheva & S. Golovanova & Y. Katsoulacos, 2019. "Optimal Institutional Structure of Competition Authorities Under Reputation Maximization: A Model and Empirical Evidence from the Case of Russia," Review of Industrial Organization, Springer;The Industrial Organization Society, vol. 54(2), pages 251-282, March.
    15. repec:mpr:mprres:2956 is not listed on IDEAS
    16. Kuhn, Michael & Siciliani, Luigi, 2007. "Performance Indicators for Quality with Adverse Selection, Gaming and Inequality Aversion," CEPR Discussion Papers 6261, C.E.P.R. Discussion Papers.
    17. Alfonso Flores‐Lagunes & Arturo Gonzalez & Todd Neumann, 2010. "Learning But Not Earning? The Impact Of Job Corps Training On Hispanic Youth," Economic Inquiry, Western Economic Association International, vol. 48(3), pages 651-667, July.
    18. Carneiro, Pedro & Heckman, James J., 2003. "Human Capital Policy," IZA Discussion Papers 821, IZA Network @ LISER.
    19. James J. Heckman & Carolyn Heinrich & Jeffrey Smith, 2002. "The Performance of Performance Standards," Journal of Human Resources, University of Wisconsin Press, vol. 37(4), pages 778-811.
    20. repec:mpr:mprres:7736 is not listed on IDEAS
    21. Aakvik, Arild & Heckman, James J. & Vytlacil, Edward J., 2005. "Estimating treatment effects for discrete outcomes when responses to treatment vary: an application to Norwegian vocational rehabilitation programs," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 15-51.
    22. Steven Raphael, 2010. "Improving Employment Prospects for Former Prison Inmates: Challenges and Policy," NBER Working Papers 15874, National Bureau of Economic Research, Inc.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:jpamgt:v:27:y:2008:i:3:p:556-576. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/journal/34787/home .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.