IDEAS home Printed from https://ideas.repec.org/a/eee/ecoedu/v89y2022ics0272775722000553.html
   My bibliography  Save this article

Do learning communities increase first year college retention? Evidence from a randomized control trial

Author

Listed:
  • Azzam, Tarek
  • Bates, Michael D.
  • Fairris, David

Abstract

In this paper, we estimate the impact of a learning community on first-year college retention at a four-year public research university using a randomized control trial (RCT) for those students who opt into the experiment. Intent-to-treat and local-average-treatment-effect estimates reveal no discernable programmatic effects. We also generate estimates of program impact using observational techniques and find estimated impacts that are positive, large and statistically significant. We explore a variety of selection processes to better understand the differences between the RCT and observational estimates and to reflect on the generalizability of the RCT results for various other populations of interest. Non-random selection into the experimental sample accounts for the major difference in the two estimates and also cautions against generalizing the RCT result for populations outside the experiment.

Suggested Citation

  • Azzam, Tarek & Bates, Michael D. & Fairris, David, 2022. "Do learning communities increase first year college retention? Evidence from a randomized control trial," Economics of Education Review, Elsevier, vol. 89(C).
    • Handle: RePEc:eee:ecoedu:v:89:y:2022:i:c:s0272775722000553
    DOI: 10.1016/j.econedurev.2022.102279
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0272775722000553
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.econedurev.2022.102279?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2009. "Dealing with limited overlap in estimation of average treatment effects," Biometrika, Biometrika Trust, vol. 96(1), pages 187-199.
    2. Bo, Hao & Galiani, Sebastian, 2021. "Assessing external validity," Research in Economics, Elsevier, vol. 75(3), pages 274-285.
    3. Amanda Kowalski, 2016. "Doing more when you're running LATE: Applying marginal treatment effect methods to examine treatment effect heterogeneity in experiments," Artefactual Field Experiments 00560, The Field Experiments Website.
    4. Black, Dan A. & Joo, Joonhwi & LaLonde, Robert & Smith, Jeffrey A. & Taylor, Evan J., 2022. "Simple Tests for Selection: Learning More from Instrumental Variables," Labour Economics, Elsevier, vol. 79(C).
    5. Yang L. & Tsiatis A. A., 2001. "Efficiency Study of Estimators for a Treatment Effect in a Pretest-Posttest Trial," The American Statistician, American Statistical Association, vol. 55, pages 314-321, November.
    6. Philip Oreopoulos & Daniel Lang & Joshua Angrist, 2009. "Incentives and Services for College Achievement: Evidence from a Randomized Trial," American Economic Journal: Applied Economics, American Economic Association, vol. 1(1), pages 136-163, January.
    7. Sianesi, Barbara, 2017. "Evidence of randomisation bias in a large-scale social experiment: The case of ERA," Journal of Econometrics, Elsevier, vol. 198(1), pages 41-64.
    8. Huber, Martin, 2013. "A simple test for the ignorability of non-compliance in experiments," Economics Letters, Elsevier, vol. 120(3), pages 389-391.
    9. Joshua D. Angrist, 2004. "Treatment effect heterogeneity in theory and practice," Economic Journal, Royal Economic Society, vol. 114(494), pages 52-83, March.
    10. Paloyo, Alfredo R. & Rogan, Sally & Siminski, Peter, 2016. "The effect of supplemental instruction on academic performance: An encouragement design experiment," Economics of Education Review, Elsevier, vol. 55(C), pages 57-69.
    11. Sebastian Galiani & Patrick J. McEwan & Brian Quistorff, 2017. "External and Internal Validity of a Geographic Quasi-Experiment Embedded in a Cluster-Randomized Experiment," Advances in Econometrics, in: Regression Discontinuity Designs, volume 38, pages 195-236, Emerald Group Publishing Limited.
    12. Jeremy Lise & Shannon Seitz & Jeffrey Smith, 2015. "Evaluating search and matching models using experimental data," IZA Journal of Labor Economics, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 4(1), pages 1-35, December.
    13. Dean Karlan & Jonathan Zinman, 2009. "Observing Unobservables: Identifying Information Asymmetries With a Consumer Credit Field Experiment," Econometrica, Econometric Society, vol. 77(6), pages 1993-2008, November.
    14. Christopher R. Walters, 2018. "The Demand for Effective Charter Schools," Journal of Political Economy, University of Chicago Press, vol. 126(6), pages 2179-2223.
    15. Julie Hotchkiss & Robert Moore & M. Melinda Pitts, 2006. "Freshman Learning Communities, College Performance, and Retention," Education Economics, Taylor & Francis Journals, vol. 14(2), pages 197-210.
    16. Hunt Allcott, 2015. "Site Selection Bias in Program Evaluation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(3), pages 1117-1165.
    17. Christian N. Brinch & Magne Mogstad & Matthew Wiswall, 2017. "Beyond LATE with a Discrete Instrument," Journal of Political Economy, University of Chicago Press, vol. 125(4), pages 985-1039.
    18. repec:mpr:mprres:6573 is not listed on IDEAS
    19. Erin Hartman & Richard Grieve & Roland Ramsahai & Jasjeet S. Sekhon, 2015. "From sample average treatment effect to population average treatment effect on the treated: combining experimental with observational studies to estimate population treatment effects," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 178(3), pages 757-778, June.
    20. Hausman, Jerry A & Wise, David A, 1979. "Attrition Bias in Experimental and Panel Data: The Gary Income Maintenance Experiment," Econometrica, Econometric Society, vol. 47(2), pages 455-473, March.
    21. Russell, Lauren, 2017. "Can learning communities boost success of women and minorities in STEM? Evidence from the Massachusetts Institute of Technology," Economics of Education Review, Elsevier, vol. 61(C), pages 98-111.
    22. Amanda E. Kowalski, 2016. "Doing More When You're Running LATE: Applying Marginal Treatment Effect Methods to Examine Treatment Effect Heterogeneity in Experiments for the Young and Privately Insured"," Cowles Foundation Discussion Papers 2045, Cowles Foundation for Research in Economics, Yale University.
    23. Onur Altindag & Theodore J. Joyce & Julie A. Reeder, 2015. "Effects of Peer Counseling to Support Breastfeeding: Assessing the External Validity of a Randomized Field Experiment," NBER Working Papers 21013, National Bureau of Economic Research, Inc.
    24. Amanda E. Kowalski, 2018. "How to Examine External Validity Within an Experiment," NBER Working Papers 24834, National Bureau of Economic Research, Inc.
    25. Wooldridge, Jeffrey M., 2014. "Quasi-maximum likelihood estimation and testing for nonlinear models with endogenous explanatory variables," Journal of Econometrics, Elsevier, vol. 182(1), pages 226-234.
    26. Edward Vytlacil, 2002. "Independence, Monotonicity, and Latent Index Models: An Equivalence Result," Econometrica, Econometric Society, vol. 70(1), pages 331-341, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kofoed, Michael S. & Jones, Todd R., 2023. "First Generation College Students and Peer Effects," IZA Discussion Papers 16198, Institute of Labor Economics (IZA).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tarek Azzam & Michael Bates & David Fairris, 2019. "Do Learning Communities Increase First Year College Retention? Testing Sample Selection and External Validity of Randomized Control Trials," Working Papers 202002, University of California at Riverside, Department of Economics.
    2. Amanda E Kowalski, 2023. "Behaviour within a Clinical Trial and Implications for Mammography Guidelines," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(1), pages 432-462.
    3. Black, Dan A. & Joo, Joonhwi & LaLonde, Robert & Smith, Jeffrey A. & Taylor, Evan J., 2022. "Simple Tests for Selection: Learning More from Instrumental Variables," Labour Economics, Elsevier, vol. 79(C).
    4. Huber, Martin & Wüthrich, Kaspar, 2017. "Evaluating local average and quantile treatment effects under endogeneity based on instruments: a review," FSES Working Papers 479, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    5. Domenico Depalo, 2020. "Explaining the causal effect of adherence to medication on cholesterol through the marginal patient," Health Economics, John Wiley & Sons, Ltd., vol. 29(S1), pages 110-126, October.
    6. Patrick Kline & Christopher R. Walters, 2019. "On Heckits, LATE, and Numerical Equivalence," Econometrica, Econometric Society, vol. 87(2), pages 677-696, March.
    7. Robert A. Moffitt & Matthew V. Zahn, 2019. "The Marginal Labor Supply Disincentives of Welfare: Evidence from Administrative Barriers to Participation," NBER Working Papers 26028, National Bureau of Economic Research, Inc.
    8. Pereda-Fernández, Santiago, 2023. "Identification and estimation of triangular models with a binary treatment," Journal of Econometrics, Elsevier, vol. 234(2), pages 585-623.
    9. Michael R.M. Abrigo & Timothy J. Halliday & Teresa Molina, 2022. "Expanding health insurance for the elderly of the Philippines," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(3), pages 500-520, April.
    10. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    11. Isaiah Andrews & Emily Oster, 2017. "A Simple Approximation for Evaluating External Validity Bias," NBER Working Papers 23826, National Bureau of Economic Research, Inc.
    12. Schmieder, Julia, 2021. "Fertility as a driver of maternal employment," Labour Economics, Elsevier, vol. 72(C).
    13. Philip Marx, 2020. "Sharp Bounds in the Latent Index Selection Model," Papers 2012.02390, arXiv.org, revised Apr 2023.
    14. Julia Schmieder, 2020. "Fertility as a Driver of Maternal Employment," Discussion Papers of DIW Berlin 1882, DIW Berlin, German Institute for Economic Research.
    15. Stephen Coussens & Jann Spiess, 2021. "Improving Inference from Simple Instruments through Compliance Estimation," Papers 2108.03726, arXiv.org.
    16. Bodory, Hugo & Huber, Martin, 2018. "The causalweight package for causal inference in R," FSES Working Papers 493, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    17. Timothy B. Armstrong & Michal Kolesár, 2021. "Sensitivity analysis using approximate moment condition models," Quantitative Economics, Econometric Society, vol. 12(1), pages 77-108, January.
    18. Huber Martin & Wüthrich Kaspar, 2019. "Local Average and Quantile Treatment Effects Under Endogeneity: A Review," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-27, January.
    19. Sokbae Lee & Bernard Salanié, 2018. "Identifying Effects of Multivalued Treatments," Econometrica, Econometric Society, vol. 86(6), pages 1939-1963, November.
    20. Jörg Peters & Jörg Langbein & Gareth Roberts, 2018. "Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity," The World Bank Research Observer, World Bank, vol. 33(1), pages 34-64.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ecoedu:v:89:y:2022:i:c:s0272775722000553. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/econedurev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.