IDEAS home Printed from https://ideas.repec.org/a/wly/jpamgt/v27y2008i3p556-576.html
   My bibliography  Save this article

Do Job Corps performance measures track program impacts?

Author

Listed:
  • Peter Z. Schochet

    (Mathematica Policy Research, Inc.)

  • John A. Burghardt

    (Mathematica Policy Research, Inc.)

Abstract

Since the 1993 Government Performance and Results Act, performance measurement systems based on short-term program outcomes have been increasingly used to assess the effectiveness of federal programs. This paper examines the association between program performance measures and long-term program impacts, using nine-year follow-up data from a recent large-scale, national experimental evaluation of Job Corps, the nation's largest federal job training program for disadvantaged youths. Job Corps is an important test case because it uses a comprehensive performance system that is widely emulated. We find that impacts on key outcomes are not associated with measured center performance levels. Participants in higherperforming centers had better outcomes; however, the same pattern holds for comparable controls. Thus, the performance measurement system is not achieving the goal of ranking and rewarding centers on the basis of their ability to improve participant outcomes relative to what these outcomes would have been otherwise. © 2008 by the Association for Public Policy Analysis and Management.

Suggested Citation

  • Peter Z. Schochet & John A. Burghardt, 2008. "Do Job Corps performance measures track program impacts?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 27(3), pages 556-576.
    • Handle: RePEc:wly:jpamgt:v:27:y:2008:i:3:p:556-576
    DOI: 10.1002/pam.20356
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1002/pam.20356
    File Function: Link to full text; subscription required
    Download Restriction: no
    File URL: https://libkey.io/10.1002/pam.20356?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. James J. Heckman & Carolyn Heinrich & Jeffrey Smith, 2002. "The Performance of Performance Standards," Journal of Human Resources, University of Wisconsin Press, vol. 37(4), pages 778-811.
    2. Peter Z. Schochet & John Burghardt & Steven Glazerman, 2001. "National Job Corps Study: The Impacts of Job Corps on Participants' Employment and Related Outcomes," Mathematica Policy Research Reports db6c4204b8e1408bb0c6289ec, Mathematica Policy Research.
    3. repec:mpr:mprres:1966 is not listed on IDEAS
    4. Cragg, Michael, 1997. "Performance Incentives in the Public Sector: Evidence from the Job Training Partnership Act," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 13(1), pages 147-168, April.
    5. Howard S. Bloom, 1984. "Accounting for No-Shows in Experimental Evaluation Designs," Evaluation Review, , vol. 8(2), pages 225-246, April.
    6. Terry Johnson & Mark Gritz & Russell Jackson & John Burghardt & Carol Boussy & Jan Leonard & Carlyn Orians, 1999. "National Job Corps Study: Report on the Process Analysis," Mathematica Policy Research Reports efc0cd05f0524a049779f797f, Mathematica Policy Research.
    7. Pascal Courty & Gerald Marschke, 2002. "Performance Incentives with Award Constraints," Journal of Human Resources, University of Wisconsin Press, vol. 37(4), pages 812-845.
    8. Sheena McConnell & Steven Glazerman, 2001. "National Job Corps Study: The Benefits and Costs of Job Corps," Mathematica Policy Research Reports 19ff8678a108410587c5dfad0, Mathematica Policy Research.
    9. Robert S. Gay & Michael E. Borus, 1980. "Validating Performance Indicators for Employment and Training Programs," Journal of Human Resources, University of Wisconsin Press, vol. 15(1), pages 29-48.
    10. repec:mpr:mprres:2951 is not listed on IDEAS
    11. Burt S Barnow, 2000. "Exploring the relationship between performance management and program impact: A case study of the job training partnership act," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 19(1), pages 118-141.
    12. repec:mpr:mprres:2955 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Joanne Lee & Peter Z. Schochet & Jillian Berk, "undated". "The External Review of Job Corps: Directions for Future Research," Mathematica Policy Research Reports 376221bbee0d4b40bda431a16, Mathematica Policy Research.
    2. Burt S. Barnow & Jeffrey Smith, 2015. "Employment and Training Programs," NBER Chapters, in: Economics of Means-Tested Transfer Programs in the United States, Volume 2, pages 127-234, National Bureau of Economic Research, Inc.
    3. Jesse Rothstein & Till von Wachter, 2016. "Social Experiments in the Labor Market," NBER Working Papers 22585, National Bureau of Economic Research, Inc.
    4. Flores-Lagunes, Alfonso & Gonzalez, Arturo & Neumann, Todd C., 2007. "Estimating the Effects of Length of Exposure to a Training Program: The Case of Job Corps," IZA Discussion Papers 2846, Institute of Labor Economics (IZA).
    5. Sameh Hallaq, 2020. "The Palestinian Labor Market over the Last Three Decades," Economics Working Paper Archive wp_976, Levy Economics Institute.
    6. Jane Fortson & Peter Z. Schochet, "undated". "Analysis of Associations Between Contemporaneous Job Corps Performance Measures and Impact Estimates from the National Job Corps Study," Mathematica Policy Research Reports 66ddefe84fb84c1e9a0666e17, Mathematica Policy Research.
    7. repec:mpr:mprres:7650 is not listed on IDEAS
    8. Douglas J. Besharov, 2009. "Presidential address: From the Great Society to continuous improvement government: Shifting from “does it work?” to “what would make it better?”," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 28(2), pages 199-220.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Burt S. Barnow & Jeffrey Smith, 2015. "Employment and Training Programs," NBER Chapters, in: Economics of Means-Tested Transfer Programs in the United States, Volume 2, pages 127-234, National Bureau of Economic Research, Inc.
    2. Robert J. LaLonde, 2003. "Employment and Training Programs," NBER Chapters, in: Means-Tested Transfer Programs in the United States, pages 517-586, National Bureau of Economic Research, Inc.
    3. Pascal Courty & Gerald Marschke, 2003. "Making Government Accountable: Lessons from a Federal Job Training Program," The Centre for Market and Public Organisation 03/083, The Centre for Market and Public Organisation, University of Bristol, UK.
    4. Courty, Pascal & Marschke, Gerald, 2004. "A General Test of Gaming," CEPR Discussion Papers 4514, C.E.P.R. Discussion Papers.
    5. Peter Z. Schochet & John Burghardt, 2007. "Using Propensity Scoring to Estimate Program-Related Subgroup Impacts in Experimental Program Evaluations," Evaluation Review, , vol. 31(2), pages 95-120, April.
    6. Peter Z. Schochet, 2021. "Long‐Run Labor Market Effects of the Job Corps Program: Evidence from a Nationally Representative Experiment," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 40(1), pages 128-157, January.
    7. Peter Z. Schochet & Ronald D'Amico & Jillian Berk & Sarah Dolfin & Nathan Wozny, "undated". "Estimated Impacts for Participants in the Trade Adjustment Assistance (TAA) Program Under the 2002 Amendments," Mathematica Policy Research Reports 582d8723f6884d4eb7a3f95a4, Mathematica Policy Research.
    8. Annalisa Mastri & Emily Evans & Stacy Pancratz & Carrie Wolfson, "undated". "Employment Research in Brief: An Annotated Bibliography of ETA-Sponsored Studies," Mathematica Policy Research Reports 2f5be979ffa14d3996aa77f33, Mathematica Policy Research.
    9. Courty, Pascal & Kim, Do Han & Marschke, Gerald, 2011. "Curbing cream-skimming: Evidence on enrolment incentives," Labour Economics, Elsevier, vol. 18(5), pages 643-655, October.
    10. James J. Heckman & Carolyn Heinrich & Jeffrey Smith, 2002. "The Performance of Performance Standards," Journal of Human Resources, University of Wisconsin Press, vol. 37(4), pages 778-811.
    11. repec:mpr:mprres:7300 is not listed on IDEAS
    12. repec:mpr:mprres:2956 is not listed on IDEAS
    13. Kuhn, Michael & Siciliani, Luigi, 2007. "Performance Indicators for Quality with Adverse Selection, Gaming and Inequality Aversion," CEPR Discussion Papers 6261, C.E.P.R. Discussion Papers.
    14. Steven Raphael, 2010. "Improving Employment Prospects for Former Prison Inmates: Challenges and Policy," NBER Working Papers 15874, National Bureau of Economic Research, Inc.
    15. Deborah A. Cobb‐Clark & Thomas Crossley, 2003. "Econometrics for Evaluations: An Introduction to Recent Developments," The Economic Record, The Economic Society of Australia, vol. 79(247), pages 491-511, December.
    16. Pascal Courty & Gerald Marschke, 2004. "An Empirical Investigation of Gaming Responses to Explicit Performance Incentives," Journal of Labor Economics, University of Chicago Press, vol. 22(1), pages 23-56, January.
    17. Jun Li, 2022. "Value‐Based Payments in Health Care: Evidence from a Nationwide Randomized Experiment in the Home Health Sector," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 41(4), pages 1090-1117, September.
    18. Ozkan Eren & Serkan Ozbeklik, 2014. "Who Benefits From Job Corps? A Distributional Analysis Of An Active Labor Market Program," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 29(4), pages 586-611, June.
    19. Heinrich Hock & Dara Lee Luca & Tim Kautz & David Stapleton, "undated". "Improving the Outcomes of Youth with Medical Limitations Through Comprehensive Training and Employment Services: Evidence from the National Job Corps Study," Mathematica Policy Research Reports 00b09009914b4c9986a85fe3e, Mathematica Policy Research.
    20. Steven Raphael, 2010. "Improving Employment Prospects for Former Prison Inmates: Challenges and Policy," NBER Chapters, in: Controlling Crime: Strategies and Tradeoffs, pages 521-565, National Bureau of Economic Research, Inc.
    21. Aakvik, Arild & Heckman, James J. & Vytlacil, Edward J., 2005. "Estimating treatment effects for discrete outcomes when responses to treatment vary: an application to Norwegian vocational rehabilitation programs," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 15-51.
    22. Gerald Marschke & Pascal Courty, 2000. "An Empirical Investigation of Gaming Responses to Performance Incentives," Discussion Papers 00-12, University at Albany, SUNY, Department of Economics.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:jpamgt:v:27:y:2008:i:3:p:556-576. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/journal/34787/home .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.