IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp18840.html

Unpacking Treatment Effect Heterogeneity in a Large-Scale Dropout-Prevention Experiment

Author

Listed:
  • Dee, Thomas

    (Stanford University)

  • Gibbs, Chloe

    (University of Notre Dame)

  • Margolit-Chan, Rebecca

    (Cornell University)

  • Morrell, Lillian

    (University of Notre Dame)

  • Sullivan, James

    (University of Notre Dame)

Abstract

Timely high school completion is a critical step in successful transitions to post-secondary schooling and the labor market. This randomized controlled trial, deployed in two urban, disadvantaged school districts, tests the effectiveness of a mature and comprehensive student-support program on an unusually large scale (n=2,528). While overall effects on graduation rates are small and statistically insignificant, exploratory analyses reveal notable variation by site and over time, mirroring heterogeneity in treatment timing and program implementation. These findings highlight the challenges of scaling interventions and raise concerns about generalizing results from small-scale, single-site evaluations.

Suggested Citation

  • Dee, Thomas & Gibbs, Chloe & Margolit-Chan, Rebecca & Morrell, Lillian & Sullivan, James, 2026. "Unpacking Treatment Effect Heterogeneity in a Large-Scale Dropout-Prevention Experiment," IZA Discussion Papers 18840, IZA Network @ LISER.
    • Handle: RePEc:iza:izadps:dp18840
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp18840.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Sara B. Heller & Anuj K. Shah & Jonathan Guryan & Jens Ludwig & Sendhil Mullainathan & Harold A. Pollack, 2017. "Thinking, Fast and Slow? Some Field Experiments to Reduce Crime and Dropout in Chicago," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(1), pages 1-54.
    2. Hunt Allcott, 2015. "Site Selection Bias in Program Evaluation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(3), pages 1117-1165.
    3. Alberto Abadie & Susan Athey & Guido W Imbens & Jeffrey M Wooldridge, 2023. "When Should You Adjust Standard Errors for Clustering?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 138(1), pages 1-35.
    4. Clément de Chaisemartin & Luc Behaghel, 2020. "Estimating the Effect of Treatments Allocated by Randomized Waiting Lists," Econometrica, Econometric Society, vol. 88(4), pages 1453-1477, July.
    5. O. Ashenfelter & D. Card (ed.), 1999. "Handbook of Labor Economics," Handbook of Labor Economics, Elsevier, edition 1, volume 3, number 3.
    6. Sara Heller & Harold A. Pollack & Roseanna Ander & Jens Ludwig, 2013. "Preventing Youth Violence and Dropout: A Randomized Field Experiment," NBER Working Papers 19014, National Bureau of Economic Research, Inc.
    7. Michael J. Weiss & Howard S. Bloom & Thomas Brock, 2014. "A Conceptual Framework For Studying The Sources Of Variation In Program Effects," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 33(3), pages 778-808, June.
    8. Julie Berry Cullen & Steven D. Levitt & Erin Robertson & Sally Sadoff, 2013. "What Can Be Done to Improve Struggling High Schools?," Journal of Economic Perspectives, American Economic Association, vol. 27(2), pages 133-152, Spring.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Christopher Blattman & Jeannie Annan, 2015. "Can Employment Reduce Lawlessness and Rebellion? A Field Experiment with High-Risk Men in a Fragile State," NBER Working Papers 21289, National Bureau of Economic Research, Inc.
    2. Laura Jaitman, 2019. "Frontiers in the economics of crime: lessons for Latin America and the Caribbean," Latin American Economic Review, Springer;Centro de Investigaciòn y Docencia Económica (CIDE), vol. 28(1), pages 1-36, December.
    3. John A. List, 2024. "Optimally generate policy-based evidence before scaling," Nature, Nature, vol. 626(7999), pages 491-499, February.
    4. Diane Alexander, 2020. "How Do Doctors Respond to Incentives? Unintended Consequences of Paying Doctors to Reduce Costs," Journal of Political Economy, University of Chicago Press, vol. 128(11), pages 4046-4096.
    5. Christopher Blattman & Julian C. Jamison & Margaret Sheridan, 2015. "Reducing crime and violence: Experimental evidence from cognitive behavioral therapy in Liberia," NBER Working Papers 21204, National Bureau of Economic Research, Inc.
    6. Benjamin Lu & Eli Ben-Michael & Avi Feller & Luke Miratrix, 2023. "Is It Who You Are or Where You Are? Accounting for Compositional Differences in Cross-Site Treatment Effect Variation," Journal of Educational and Behavioral Statistics, , vol. 48(4), pages 420-453, August.
    7. Alison Evans Cuellar & Dhaval M. Dave, 2015. "Causal Effects of Mental Health Treatment on Education Outcomes for Youth in the Justice System," NBER Working Papers 21206, National Bureau of Economic Research, Inc.
    8. Friebel, Guido & Manchin, Miriam & Mendola, Mariapia & Prarolo, Giovanni, 2018. "International Migration Intentions and Illegal Costs: Evidence from Africa-to-Europe Smuggling Routes," IZA Discussion Papers 11978, IZA Network @ LISER.
    9. Stephen Machin & Matteo Sandi, 2025. "Crime and Education," Annual Review of Economics, Annual Reviews, vol. 17(1), pages 241-260, August.
    10. Chowdhury, Shyamal & Hasan, Syed & Sharma, Uttam, 2024. "The Role of Trainee Selection in the Effectiveness of Vocational Training: Evidence from a Randomized Controlled Trial in Nepal," IZA Discussion Papers 16705, IZA Network @ LISER.
    11. Damm, Anna Piil & Hassani, Ahmad & Sørensen, Jonas Søndergaard, 2025. "Place-Based Policies in Deprived Neighbourhoods: Opportunities for Preexisting Residents and Neighbourhood Revitalisation?," IZA Discussion Papers 17843, IZA Network @ LISER.
    12. David J. Deming & Mikko I. Silliman, 2024. "Skills and Human Capital in the Labor Market," NBER Working Papers 32908, National Bureau of Economic Research, Inc.
    13. Analia Schlosser & Yannay Shanan, 2025. "Fostering Soft Skills in Active Labor Market Programs: Evidence from a Large-Scale RCT," Journal of Human Resources, University of Wisconsin Press, vol. 60(1), pages 1-36.
    14. Le Barbanchon, Thomas & Ubfal, Diego & Araya, Federico, 2020. "The Effects of Working While in School: Evidence from Uruguayan Lotteries," IZA Discussion Papers 13929, IZA Network @ LISER.
    15. Jens Dietrichson & Trine Filges & Rasmus H. Klokker & Bjørn C. A. Viinholt & Martin Bøg & Ulla H. Jensen, 2020. "Targeted school‐based interventions for improving reading and mathematics for students with, or at risk of, academic difficulties in Grades 7–12: A systematic review," Campbell Systematic Reviews, John Wiley & Sons, vol. 16(2), June.
    16. Howlader, Aparna & Ando, Amy W., 2016. "Consequences of Protected Areas for Forest Extraction and Human Well-being: Evidence from Nepal," 2016 Annual Meeting, July 31-August 2, Boston, Massachusetts 236272, Agricultural and Applied Economics Association.
    17. Esterling, Kevin & Brady, David & Schwitzgebel, Eric, 2021. "The Necessity of Construct and External Validity for Generalized Causal Claims," OSF Preprints 2s8w5, Center for Open Science.
    18. Ek, Claes & Söderberg, Magnus, 2024. "Norm-based feedback on household waste: Large-scale field experiments in two Swedish municipalities," Journal of Public Economics, Elsevier, vol. 238(C).
    19. Seth Gershenson & Cassandra M. D. Hart & Joshua Hyman & Constance A. Lindsay & Nicholas W. Papageorge, 2022. "The Long-Run Impacts of Same-Race Teachers," American Economic Journal: Economic Policy, American Economic Association, vol. 14(4), pages 300-342, November.
    20. Brody, Ilana & Dai, Hengchen & Saccardo, Silvia & Milkman, Katherine L. & Duckworth, Angela L. & Patel, Mitesh S. & Gromet, Dena M., 2026. "Targeting behavioral interventions based on past behavior: Evidence from vaccine uptake," Organizational Behavior and Human Decision Processes, Elsevier, vol. 192(C).

    More about this item

    Keywords

    ;
    ;

    JEL classification:

    • I24 - Health, Education, and Welfare - - Education - - - Education and Inequality
    • I26 - Health, Education, and Welfare - - Education - - - Returns to Education
    • I31 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - General Welfare, Well-Being

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp18840. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Mark Fallak (email available below). General contact details of provider: https://edirc.repec.org/data/izaaalu.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.