IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp18840.html

Unpacking Treatment Effect Heterogeneity in a Large-Scale Dropout-Prevention Experiment

Author

Listed:
  • Dee, Thomas

    (Stanford University)

  • Gibbs, Chloe

    (University of Notre Dame)

  • Margolit-Chan, Rebecca

    (Cornell University)

  • Morrell, Lillian

    (University of Notre Dame)

  • Sullivan, James

    (University of Notre Dame)

Abstract

Timely high school completion is a critical step in successful transitions to post-secondary schooling and the labor market. This randomized controlled trial, deployed in two urban, disadvantaged school districts, tests the effectiveness of a mature and comprehensive student-support program on an unusually large scale (n=2,528). While overall effects on graduation rates are small and statistically insignificant, exploratory analyses reveal notable variation by site and over time, mirroring heterogeneity in treatment timing and program implementation. These findings highlight the challenges of scaling interventions and raise concerns about generalizing results from small-scale, single-site evaluations.

Suggested Citation

  • Dee, Thomas & Gibbs, Chloe & Margolit-Chan, Rebecca & Morrell, Lillian & Sullivan, James, 2026. "Unpacking Treatment Effect Heterogeneity in a Large-Scale Dropout-Prevention Experiment," IZA Discussion Papers 18840, IZA Network @ LISER.
    • Handle: RePEc:iza:izadps:dp18840
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp18840.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Sara B. Heller & Anuj K. Shah & Jonathan Guryan & Jens Ludwig & Sendhil Mullainathan & Harold A. Pollack, 2017. "Thinking, Fast and Slow? Some Field Experiments to Reduce Crime and Dropout in Chicago," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(1), pages 1-54.
    2. Hunt Allcott, 2015. "Site Selection Bias in Program Evaluation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(3), pages 1117-1165.
    3. Alberto Abadie & Susan Athey & Guido W Imbens & Jeffrey M Wooldridge, 2023. "When Should You Adjust Standard Errors for Clustering?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 138(1), pages 1-35.
    4. Clément de Chaisemartin & Luc Behaghel, 2020. "Estimating the Effect of Treatments Allocated by Randomized Waiting Lists," Econometrica, Econometric Society, vol. 88(4), pages 1453-1477, July.
    5. O. Ashenfelter & D. Card (ed.), 1999. "Handbook of Labor Economics," Handbook of Labor Economics, Elsevier, edition 1, volume 3, number 3.
    6. Sara Heller & Harold A. Pollack & Roseanna Ander & Jens Ludwig, 2013. "Preventing Youth Violence and Dropout: A Randomized Field Experiment," NBER Working Papers 19014, National Bureau of Economic Research, Inc.
    7. Julie Berry Cullen & Steven D. Levitt & Erin Robertson & Sally Sadoff, 2013. "What Can Be Done to Improve Struggling High Schools?," Journal of Economic Perspectives, American Economic Association, vol. 27(2), pages 133-152, Spring.
    8. Michael J. Weiss & Howard S. Bloom & Thomas Brock, 2014. "A Conceptual Framework For Studying The Sources Of Variation In Program Effects," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 33(3), pages 778-808, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Christopher Blattman & Jeannie Annan, 2015. "Can Employment Reduce Lawlessness and Rebellion? A Field Experiment with High-Risk Men in a Fragile State," NBER Working Papers 21289, National Bureau of Economic Research, Inc.
    2. Laura Jaitman, 2019. "Frontiers in the economics of crime: lessons for Latin America and the Caribbean," Latin American Economic Review, Springer;Centro de Investigaciòn y Docencia Económica (CIDE), vol. 28(1), pages 1-36, December.
    3. John A. List, 2024. "Optimally generate policy-based evidence before scaling," Nature, Nature, vol. 626(7999), pages 491-499, February.
    4. Benjamin Lu & Eli Ben-Michael & Avi Feller & Luke Miratrix, 2023. "Is It Who You Are or Where You Are? Accounting for Compositional Differences in Cross-Site Treatment Effect Variation," Journal of Educational and Behavioral Statistics, , vol. 48(4), pages 420-453, August.
    5. Alison Evans Cuellar & Dhaval M. Dave, 2015. "Causal Effects of Mental Health Treatment on Education Outcomes for Youth in the Justice System," NBER Working Papers 21206, National Bureau of Economic Research, Inc.
    6. Damm, Anna Piil & Hassani, Ahmad & Sørensen, Jonas Søndergaard, 2025. "Place-Based Policies in Deprived Neighbourhoods: Opportunities for Preexisting Residents and Neighbourhood Revitalisation?," IZA Discussion Papers 17843, IZA Network @ LISER.
    7. Analia Schlosser & Yannay Shanan, 2025. "Fostering Soft Skills in Active Labor Market Programs: Evidence from a Large-Scale RCT," Journal of Human Resources, University of Wisconsin Press, vol. 60(1), pages 1-36.
    8. Howlader, Aparna & Ando, Amy W., 2016. "Consequences of Protected Areas for Forest Extraction and Human Well-being: Evidence from Nepal," 2016 Annual Meeting, July 31-August 2, Boston, Massachusetts 236272, Agricultural and Applied Economics Association.
    9. Seth Gershenson & Cassandra M. D. Hart & Joshua Hyman & Constance A. Lindsay & Nicholas W. Papageorge, 2022. "The Long-Run Impacts of Same-Race Teachers," American Economic Journal: Economic Policy, American Economic Association, vol. 14(4), pages 300-342, November.
    10. Brody, Ilana & Dai, Hengchen & Saccardo, Silvia & Milkman, Katherine L. & Duckworth, Angela L. & Patel, Mitesh S. & Gromet, Dena M., 2026. "Targeting behavioral interventions based on past behavior: Evidence from vaccine uptake," Organizational Behavior and Human Decision Processes, Elsevier, vol. 192(C).
    11. Hönig, Tillman, 2017. "The Impact of Peace: Evidence from Nigeria," MPRA Paper 83302, University Library of Munich, Germany.
    12. Camargo Juliana & Lima Lycia & Riva Flavio & Souza André Portela, 2021. "Technical Education, Non-cognitive Skills and Labor Market Outcomes: Experimental Evidence from Brazil," IZA Journal of Labor Economics, Paradigm & Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 10(1), pages 1-34, January.
    13. Anna Adamecz & Morag Henderson & Nikki Shure, 2024. "Intergenerational educational mobility – The role of non-cognitive skills," Education Economics, Taylor & Francis Journals, vol. 32(1), pages 59-78, January.
    14. Tillman Hönig, 2019. "The Impact of Peace: Evidence from Nigeria," HiCN Working Papers 293, Households in Conflict Network.
    15. Guido Friebel & Miriam Manchin & Mariapia Mendola & Giovanni Prarolo, 2018. "International Migration Intentions and Illegal Costs: Evidence from Africa-to-Europe Smuggling Routes," Working Papers 393, University of Milano-Bicocca, Department of Economics, revised Nov 2018.
    16. Philip J. Cook & Kenneth Dodge & George Farkas & Roland G. Fryer, Jr & Jonathan Guryan & Jens Ludwig & Susan Mayer & Harold Pollack & Laurence Steinberg, 2014. "The (Surprising) Efficacy of Academic and Behavioral Intervention with Disadvantaged Youth: Results from a Randomized Experiment in Chicago," NBER Working Papers 19862, National Bureau of Economic Research, Inc.
    17. Fujimoto, Junichi & Meng, Xiangcai, 2019. "Curse or blessing: Investigating the education and income of firstborns and only boys," Journal of the Japanese and International Economies, Elsevier, vol. 53(C), pages 1-1.
    18. Stephen Machin & Matteo Sandi, 2025. "Crime and Education," Annual Review of Economics, Annual Reviews, vol. 17(1), pages 241-260, August.
    19. Franziska Hampf, 2019. "The Effect of Compulsory Schooling on Skills: Evidence from a Reform in Germany," ifo Working Paper Series 313, ifo Institute - Leibniz Institute for Economic Research at the University of Munich.
    20. Diane Alexander, 2020. "How Do Doctors Respond to Incentives? Unintended Consequences of Paying Doctors to Reduce Costs," Journal of Political Economy, University of Chicago Press, vol. 128(11), pages 4046-4096.

    More about this item

    Keywords

    ;
    ;

    JEL classification:

    • I24 - Health, Education, and Welfare - - Education - - - Education and Inequality
    • I26 - Health, Education, and Welfare - - Education - - - Returns to Education
    • I31 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - General Welfare, Well-Being

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp18840. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Mark Fallak (email available below). General contact details of provider: https://edirc.repec.org/data/izaaalu.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.