IDEAS home Printed from https://ideas.repec.org/a/wly/jpamgt/v40y2021i1p158-190.html
   My bibliography  Save this article

Unpacking Complex Mediation Mechanisms And Their Heterogeneity Between Sites In A Job Corps Evaluation

Author

Listed:
  • Xu Qin
  • Jonah Deutsch
  • Guanglei Hong

Abstract

This study aims to test the theory underlying Job Corps, one of the largest education and training programs in the U.S. serving disadvantaged youth. Central to the program are vocational training and general education that serve as two concurrent mediators transmitting the program impact on earnings. To distinguish the relative contribution of each, we develop methods for decomposing the Job Corps impact on earnings into an indirect effect transmitted through vocational training, an indirect effect transmitted through general education, and a direct effect attributable to supplementary services. We further ask whether general education and vocational training reinforce each other and produce a joint impact greater than the sum of the two separate pathways. Moreover, we examine the heterogeneity of each causal effect across all the Job Corps centers. This article presents concepts and methods for defining, identifying, and estimating not only the population averages but also the between‐site variance of these causal effects. Our analytic procedure incorporates a series of weighting strategies to enhance the internal and external validity of the results and assesses the sensitivity to potential violations of the identification assumptions.

Suggested Citation

  • Xu Qin & Jonah Deutsch & Guanglei Hong, 2021. "Unpacking Complex Mediation Mechanisms And Their Heterogeneity Between Sites In A Job Corps Evaluation," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 40(1), pages 158-190, January.
  • Handle: RePEc:wly:jpamgt:v:40:y:2021:i:1:p:158-190
    DOI: 10.1002/pam.22268
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/pam.22268
    Download Restriction: no

    File URL: https://libkey.io/10.1002/pam.22268?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Hansen, Lars Peter, 1982. "Large Sample Properties of Generalized Method of Moments Estimators," Econometrica, Econometric Society, vol. 50(4), pages 1029-1054, July.
    2. Andrea Ichino & Fabrizia Mealli & Tommaso Nannicini, 2008. "From temporary help jobs to permanent employment: what can we learn from matching estimators and their sensitivity?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 23(3), pages 305-327.
    3. Martin Huber, 2014. "Identifying Causal Mechanisms (Primarily) Based On Inverse Probability Weighting," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 29(6), pages 920-943, September.
    4. Eric A. Hanushek & Guido Schwerdt & Ludger Woessmann & Lei Zhang, 2017. "General Education, Vocational Education, and Labor-Market Outcomes over the Lifecycle," Journal of Human Resources, University of Wisconsin Press, vol. 52(1), pages 48-87.
    5. Hong, Guanglei & Raudenbush, Stephen W., 2006. "Evaluating Kindergarten Retention Policy: A Case Study of Causal Inference for Multilevel Observational Data," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 901-910, September.
    6. Jeffrey R Kling & Jeffrey B Liebman & Lawrence F Katz, 2007. "Experimental Analysis of Neighborhood Effects," Econometrica, Econometric Society, vol. 75(1), pages 83-119, January.
    7. Carlos A. Flores & Alfonso Flores-Lagunes, 2013. "Partial Identification of Local Average Treatment Effects With an Invalid Instrument," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 31(4), pages 534-545, October.
    8. Susan M. Shortreed & Ashkan Ertefaie, 2017. "Outcome‐adaptive lasso: Variable selection for causal inference," Biometrics, The International Biometric Society, vol. 73(4), pages 1111-1122, December.
    9. repec:mpr:mprres:2950 is not listed on IDEAS
    10. repec:mpr:mprres:5840 is not listed on IDEAS
    11. repec:mpr:mprres:1966 is not listed on IDEAS
    12. Gary S. Becker, 1964. "Human Capital: A Theoretical and Empirical Analysis with Special Reference to Education, First Edition," NBER Books, National Bureau of Economic Research, Inc, number beck-5, March.
    13. Zimmermann, Klaus F. & Biavaschi, Costanza & Eichhorst, Werner & Giulietti, Corrado & Kendzia, Michael J. & Muravyev, Alexander & Pieters, Janneke & Rodríguez-Planas, Núria & Schmidl, Ricarda, 2013. "Youth Unemployment and Vocational Training," Foundations and Trends(R) in Microeconomics, now publishers, vol. 9(1–2), pages 1-157, December.
    14. repec:mpr:mprres:2952 is not listed on IDEAS
    15. Peter Z. Schochet & John Burghardt & Sheena McConnell, 2008. "Does Job Corps Work? Impact Findings from the National Job Corps Study," American Economic Review, American Economic Association, vol. 98(5), pages 1864-1886, December.
    16. repec:mpr:mprres:1968 is not listed on IDEAS
    17. Michael J. Weiss & Howard S. Bloom & Thomas Brock, 2014. "A Conceptual Framework For Studying The Sources Of Variation In Program Effects," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 33(3), pages 778-808, June.
    18. repec:mpr:mprres:2949 is not listed on IDEAS
    19. Xu Qin & Guanglei Hong & Jonah Deutsch & Edward Bein, 2019. "Multisite causal mediation analysis in the presence of complex sample and survey designs and non‐random non‐response," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 182(4), pages 1343-1370, October.
    20. R. M. Daniel & B. L. De Stavola & S. N. Cousens & S. Vansteelandt, 2015. "Causal mediation analysis with multiple mediators," Biometrics, The International Biometric Society, vol. 71(1), pages 1-14, March.
    21. Richard Blundell & Lorraine Dearden & Costas Meghir & Barbara Sianesi, 1999. "Human capital investment: the returns from education and training to the individual, the firm and the economy," Fiscal Studies, Institute for Fiscal Studies, vol. 20(1), pages 1-23, March.
    22. John Burghardt & Peter Z. Schochet, 2001. "National Job Corps Study: Impacts by Center Characteristics," Mathematica Policy Research Reports 79dc6705a88648e3881115389, Mathematica Policy Research.
    23. Newey, Whitney K., 1984. "A method of moments interpretation of sequential estimators," Economics Letters, Elsevier, vol. 14(2-3), pages 201-206.
    24. Terry Johnson & R. Mark Gritz & Mary Kay Dugan, 2000. "National Job Corps Study: Job Corps Applicants' Programmatic Experiences," Mathematica Policy Research Reports d3b2a2ea7ff644d7a6f8961c9, Mathematica Policy Research.
    25. Howard S. Bloom & Rebecca Unterman & Pei Zhu & Sean F. Reardon, 2020. "Lessons from New York City's Small Schools of Choice about High School Features that Promote Graduation for Disadvantaged Students," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(3), pages 740-771, June.
    26. repec:mpr:mprres:2848 is not listed on IDEAS
    27. repec:mpr:mprres:6097 is not listed on IDEAS
    28. Peter Z. Schochet, 2001. "National Job Corps Study: Methodological Appendixes on the Impact Analysis," Mathematica Policy Research Reports c3abb7b819cd4bc5a09a865d6, Mathematica Policy Research.
    29. Brian K Lee & Justin Lessler & Elizabeth A Stuart, 2011. "Weight Trimming and Propensity Score Weighting," PLOS ONE, Public Library of Science, vol. 6(3), pages 1-6, March.
    30. Jeffrey M. Albert & Suchitra Nelson, 2011. "Generalized Causal Mediation Analysis," Biometrics, The International Biometric Society, vol. 67(3), pages 1028-1038, September.
    31. Imai, Kosuke & Yamamoto, Teppei, 2013. "Identification and Sensitivity Analysis for Multiple Causal Mechanisms: Revisiting Evidence from Framing Experiments," Political Analysis, Cambridge University Press, vol. 21(2), pages 141-171, April.
    32. Terry Johnson & Mark Gritz & Russell Jackson & John Burghardt & Carol Boussy & Jan Leonard & Carlyn Orians, 1999. "National Job Corps Study: Report on the Process Analysis," Mathematica Policy Research Reports efc0cd05f0524a049779f797f, Mathematica Policy Research.
    33. Xavier De Luna & Ingeborg Waernbaum & Thomas S. Richardson, 2011. "Covariate selection for the nonparametric estimation of an average treatment effect," Biometrika, Biometrika Trust, vol. 98(4), pages 861-875.
    34. repec:mpr:mprres:2951 is not listed on IDEAS
    35. Yona Rubinstein & James J. Heckman, 2001. "The Importance of Noncognitive Skills: Lessons from the GED Testing Program," American Economic Review, American Economic Association, vol. 91(2), pages 145-149, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Guanglei Hong & Fan Yang & Xu Qin, 2023. "Posttreatment confounding in causal mediation studies: A cutting‐edge problem and a novel solution via sensitivity analysis," Biometrics, The International Biometric Society, vol. 79(2), pages 1042-1056, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xu Qin & Guanglei Hong, 2017. "A Weighting Method for Assessing Between-Site Heterogeneity in Causal Mediation Mechanism," Journal of Educational and Behavioral Statistics, , vol. 42(3), pages 308-340, June.
    2. Martin Huber & Yu‐Chin Hsu & Ying‐Ying Lee & Layal Lettry, 2020. "Direct and indirect effects of continuous treatments based on generalized propensity score weighting," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(7), pages 814-840, November.
    3. Guanglei Hong & Fan Yang & Xu Qin, 2023. "Posttreatment confounding in causal mediation studies: A cutting‐edge problem and a novel solution via sensitivity analysis," Biometrics, The International Biometric Society, vol. 79(2), pages 1042-1056, June.
    4. Huber, Martin, 2012. "Identifying causal mechanisms in experiments (primarily) based on inverse probability weighting," Economics Working Paper Series 1213, University of St. Gallen, School of Economics and Political Science, revised May 2013.
    5. Markus Frölich & Martin Huber, 2017. "Direct and indirect treatment effects–causal chains and mediation analysis with instrumental variables," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(5), pages 1645-1666, November.
    6. Martin Huber & Michael Lechner & Giovanni Mellace, 2016. "The Finite Sample Performance of Estimators for Mediation Analysis Under Sequential Conditional Independence," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(1), pages 139-160, January.
    7. Soojin Park & Peter M. Steiner & David Kaplan, 2018. "Identification and Sensitivity Analysis for Average Causal Mediation Effects with Time-Varying Treatments and Mediators: Investigating the Underlying Mechanisms of Kindergarten Retention Policy," Psychometrika, Springer;The Psychometric Society, vol. 83(2), pages 298-320, June.
    8. Helmut Farbmacher & Martin Huber & Lukáš Lafférs & Henrika Langen & Martin Spindler, 2022. "Causal mediation analysis with double machine learning [Mediation analysis via potential outcomes models]," The Econometrics Journal, Royal Economic Society, vol. 25(2), pages 277-300.
    9. Amin, Vikesh & Flores, Carlos A. & Flores-Lagunes, Alfonso & Parisian, Daniel J., 2016. "The effect of degree attainment on arrests: Evidence from a randomized social experiment," Economics of Education Review, Elsevier, vol. 54(C), pages 259-273.
    10. Flores-Lagunes, Alfonso & Gonzalez, Arturo & Neumann, Todd C., 2007. "Estimating the Effects of Length of Exposure to a Training Program: The Case of Job Corps," IZA Discussion Papers 2846, Institute of Labor Economics (IZA).
    11. Chiara Natalie Focacci, 2020. "“You reap what you sow”: Do active labour market policies always increase job security? Evidence from the Youth Guarantee," European Journal of Law and Economics, Springer, vol. 49(3), pages 373-429, June.
    12. Martin Huber, 2015. "Causal Pitfalls in the Decomposition of Wage Gaps," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 33(2), pages 179-191, April.
    13. German Blanco & Xuan Chen & Carlos A. Flores & Alfonso Flores-Lagunes, 2020. "Bounds on Average and Quantile Treatment Effects on Duration Outcomes Under Censoring, Selection, and Noncompliance," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(4), pages 901-920, October.
    14. Guanglei Hong & Fan Yang & Xu Qin, 2021. "Did you conduct a sensitivity analysis? A new weighting‐based approach for evaluations of the average treatment effect for the treated," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(1), pages 227-254, January.
    15. R. Mark Gritz & Terry Johnson, "undated". "National Job Corps Study: Assessing Program Effects on Earnings for Students Achieving Key Program Milestones," Mathematica Policy Research Reports cd3931e24b8749679a34a733b, Mathematica Policy Research.
    16. Hsu Yu-Chin & Huber Martin & Lai Tsung-Chih, 2019. "Nonparametric estimation of natural direct and indirect effects based on inverse probability weighting," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-20, January.
    17. Martin Huber & Mark Schelker & Anthony Strittmatter, 2022. "Direct and Indirect Effects based on Changes-in-Changes," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 40(1), pages 432-443, January.
    18. Peter Z. Schochet & John Burghardt, 2007. "Using Propensity Scoring to Estimate Program-Related Subgroup Impacts in Experimental Program Evaluations," Evaluation Review, , vol. 31(2), pages 95-120, April.
    19. Tyler J. VanderWeele & Eric J. Tchetgen Tchetgen, 2017. "Mediation analysis with time varying exposures and mediators," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(3), pages 917-938, June.
    20. Heinrich Hock & Dara Lee Luca & Tim Kautz & David Stapleton, "undated". "Improving the Outcomes of Youth with Medical Limitations Through Comprehensive Training and Employment Services: Evidence from the National Job Corps Study," Mathematica Policy Research Reports 00b09009914b4c9986a85fe3e, Mathematica Policy Research.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:jpamgt:v:40:y:2021:i:1:p:158-190. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/journal/34787/home .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.