IDEAS home Printed from
   My bibliography  Save this article

Evaluating the Effect of Training on Wages in the Presence of Noncompliance, Nonemployment, and Missing Outcome Data


  • Paolo Frumento
  • Fabrizia Mealli
  • Barbara Pacini
  • Donald B. Rubin


The effects of a job training program, Job Corps, on both employment and wages are evaluated using data from a randomized study. Principal stratification is used to address, simultaneously, the complications of noncompliance, wages that are only partially defined because of nonemployment, and unintended missing outcomes. The first two complications are of substantive interest, whereas the third is a nuisance. The objective is to find a parsimonious model that can be used to inform public policy. We conduct a likelihood-based analysis using finite mixture models estimated by the expectation-maximization (EM) algorithm. We maintain an exclusion restriction assumption for the effect of assignment on employment and wages for noncompliers, but not on missingness. We provide estimates under the “missing at random” assumption, and assess the robustness of our results to deviations from it. The plausibility of meaningful restrictions is investigated by means of scaled log-likelihood ratio statistics. Substantive conclusions include the following. For compliers, the effect on employment is negative in the short term; it becomes positive in the long term, but these effects are small at best. For always employed compliers, that is, compliers who are employed whether trained or not trained, positive effects on wages are found at all time periods. Our analysis reveals that background characteristics of individuals differ markedly across the principal strata. We found evidence that the program should have been better targeted, in the sense of being designed differently for different groups of people, and specific suggestions are offered. Previous analyses of this dataset, which did not address all complications in a principled manner, led to less nuanced conclusions about Job Corps.

Suggested Citation

  • Paolo Frumento & Fabrizia Mealli & Barbara Pacini & Donald B. Rubin, 2012. "Evaluating the Effect of Training on Wages in the Presence of Noncompliance, Nonemployment, and Missing Outcome Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(498), pages 450-466, June.
  • Handle: RePEc:taf:jnlasa:v:107:y:2012:i:498:p:450-466
    DOI: 10.1080/01621459.2011.643719

    Download full text from publisher

    File URL:
    Download Restriction: Access to full text is restricted to subscribers.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    1. Peter Z. Schochet, 2001. "National Job Corps Study: Methodological Appendixes on the Impact Analysis," Mathematica Policy Research Reports c3abb7b819cd4bc5a09a865d6, Mathematica Policy Research.
    2. repec:mpr:mprres:2949 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Zhichao Jiang & Peng Ding & Zhi Geng, 2016. "Principal causal effect identification and surrogate end point evaluation by multiple trials," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(4), pages 829-848, September.
    2. Mealli Fabrizia & Mattei Alessandra, 2012. "A Refreshing Account of Principal Stratification," The International Journal of Biostatistics, De Gruyter, vol. 8(1), pages 1-19, April.
    3. Markus Frölich & Martin Huber, 2014. "Treatment Evaluation With Multiple Outcome Periods Under Endogeneity and Attrition," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(508), pages 1697-1711, December.
    4. Alessandra Mattei & Fabrizia Mealli & Barbara Pacini, 2014. "Identification of causal effects in the presence of nonignorable missing outcome values," Biometrics, The International Biometric Society, vol. 70(2), pages 278-288, June.
    5. Bia, Michela & Flores-Lagunes, Alfonso & Mercatanti, Andrea, 2018. "Evaluation of Language Training Programs in Luxembourg using Principal Stratification," GLO Discussion Paper Series 289, Global Labor Organization (GLO).
    6. Huber, Martin & Wüthrich, Kaspar, 2017. "Evaluating local average and quantile treatment effects under endogeneity based on instruments: a review," FSES Working Papers 479, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    7. Chen, Xuan & Flores, Carlos A. & Flores-Lagunes, Alfonso, 2015. "Going Beyond LATE: Bounding Average Treatment Effects of Job Corps Training," IZA Discussion Papers 9511, Institute of Labor Economics (IZA).
    8. Fan Yang & Peng Ding, 2018. "Using survival information in truncation by death problems without the monotonicity assumption," Biometrics, The International Biometric Society, vol. 74(4), pages 1232-1239, December.
    9. Huber Martin & Wüthrich Kaspar, 2019. "Local Average and Quantile Treatment Effects Under Endogeneity: A Review," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-27, January.
    10. Peng Ding & Jiannan Lu, 2017. "Principal stratification analysis using principal scores," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(3), pages 757-777, June.
    11. Fan Yang & Dylan S. Small, 2016. "Using post-outcome measurement information in censoring-by-death problems," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(1), pages 299-318, January.
    12. Myoung-jae Lee, 2017. "Extensive and intensive margin effects in sample selection models: racial effects on wages," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(3), pages 817-839, June.
    13. Brian L. Egleston & Robert G. Uzzo & Yu-Ning Wong, 2017. "Latent Class Survival Models Linked by Principal Stratification to Investigate Heterogenous Survival Subgroups Among Individuals With Early-Stage Kidney Cancer," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(518), pages 534-546, April.
    14. Dalla-Zuanna, Antonio & Liu, Kai, 2019. "Understanding Program Complementarities: Estimating the Dynamic Effects of a Training Program with Multiple Alternatives," IZA Discussion Papers 12839, Institute of Labor Economics (IZA).

    More about this item


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:jnlasa:v:107:y:2012:i:498:p:450-466. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Chris Longhurst). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.