IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v37y2013i5p370-404.html
   My bibliography  Save this article

Evaluating Effects of Developmental Education for College Students Using a Regression Discontinuity Design

Author

Listed:
  • Brian G. Moss
  • William H. Yeaton

Abstract

Background: Annually, American colleges and universities provide developmental education (DE) to millions of underprepared students; however, evaluation estimates of DE benefits have been mixed. Objectives: Using a prototypic exemplar of DE, our primary objective was to investigate the utility of a replicative evaluative framework for assessing program effectiveness. Research design: Within the context of the regression discontinuity (RD) design, this research examined the effectiveness of a DE program for five, sequential cohorts of first-time college students. Discontinuity estimates were generated for individual terms and cumulatively, across terms. Subjects: Participants were 3,589 first-time community college students. Measures: DE program effects were measured by contrasting both college-level English grades and a dichotomous measure of pass/fail, for DE and non-DE students. Results: Parametric and nonparametric estimates of overall effect were positive for continuous and dichotomous measures of achievement (grade and pass/fail). The variability of program effects over time was determined by tracking results within individual terms and cumulatively, across terms. Applying this replication strategy, DE’s overall impact was modest (an effect size of approximately .20) but quite consistent, based on parametric and nonparametric estimation approaches. A meta-analysis of five RD results yielded virtually the same estimate as the overall, parametric findings. Subset analysis, though tentative, suggested that males benefited more than females, while academic gains were comparable for different ethnicities. Conclusion: The cumulative, within-study comparison, replication approach offers considerable potential for the evaluation of new and existing policies, particularly when effects are relatively small, as is often the case in applied settings.

Suggested Citation

  • Brian G. Moss & William H. Yeaton, 2013. "Evaluating Effects of Developmental Education for College Students Using a Regression Discontinuity Design," Evaluation Review, , vol. 37(5), pages 370-404, October.
    • Handle: RePEc:sae:evarev:v:37:y:2013:i:5:p:370-404
    DOI: 10.1177/0193841X14523620
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X14523620
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X14523620?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Paco Martorell & Isaac McFarlin, 2011. "Help or Hindrance? The Effects of College Remediation on Academic and Labor Market Outcomes," The Review of Economics and Statistics, MIT Press, vol. 93(2), pages 436-454, May.
    2. McCrary, Justin, 2008. "Manipulation of the running variable in the regression discontinuity design: A density test," Journal of Econometrics, Elsevier, vol. 142(2), pages 698-714, February.
    3. repec:mpr:mprres:6683 is not listed on IDEAS
    4. Ou, Dongshu, 2010. "To leave or not to leave? A regression discontinuity analysis of the impact of failing the high school exit exam," Economics of Education Review, Elsevier, vol. 29(2), pages 171-186, April.
    5. Juan Carlos Calcagno & Bridget Terry Long, 2008. "The Impact of Postsecondary Remediation Using a Regression Discontinuity Approach: Addressing Endogenous Sorting and Noncompliance," NBER Working Papers 14194, National Bureau of Economic Research, Inc.
    6. Guido Imbens & Karthik Kalyanaraman, 2012. "Optimal Bandwidth Choice for the Regression Discontinuity Estimator," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 79(3), pages 933-959.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Brian G. Moss & Peter Riley Bahr & Leigh Arsenault & Meghan Oster, 2019. "Knowing is Half the Battle, or Is It? A Randomized Experiment of the Impact of Supplemental Notification Letters on Placement Exam Participation, Preparation, and Performance," Research in Higher Education, Springer;Association for Institutional Research, vol. 60(6), pages 737-759, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michal Kurlaender & Lester Lusher & Matthew Case, 2020. "Is Early Start a Better Start? Evaluating California State University's Early Start Remediation Policy," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 348-375, March.
    2. Kalena E. Cortes & Joshua S. Goodman & Takako Nomi, 2015. "Intensive Math Instruction and Educational Attainment: Long-Run Impacts of Double-Dose Algebra," Journal of Human Resources, University of Wisconsin Press, vol. 50(1), pages 108-158.
    3. Cortes, Kalena E. & Goodman, Joshua Samuel & Nomi, Takako, 2015. "Intensive Math Instruction and Educational Attainment," Scholarly Articles 34298862, Harvard Kennedy School of Government.
    4. Jason M. Lindo & Nicholas J. Sanders & Philip Oreopoulos, 2010. "Ability, Gender, and Performance Standards: Evidence from Academic Probation," American Economic Journal: Applied Economics, American Economic Association, vol. 2(2), pages 95-117, April.
    5. De Paola, Maria & Scoppa, Vincenzo, 2015. "Procrastination, academic success and the effectiveness of a remedial program," Journal of Economic Behavior & Organization, Elsevier, vol. 115(C), pages 217-236.
    6. Steven F. Koch & Jeffrey S. Racine, 2016. "Healthcare facility choice and user fee abolition: regression discontinuity in a multinomial choice setting," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 179(4), pages 927-950, October.
    7. Hodara, Michelle & Xu, Di, 2018. "Are two subjects better than one? The effects of developmental English courses on language minority and native English-speaking students’ community college outcomes," Economics of Education Review, Elsevier, vol. 66(C), pages 1-13.
    8. Thomas J. Kane & Angela Boatman & Whitney Kozakowski & Christopher Bennett & Rachel Hitch & Dana Weisenfeld, 2021. "Is College Remediation a Barrier or a Boost? Evidence from the Tennessee SAILS Program," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 40(3), pages 883-913, June.
    9. Tafreschi, Darjusch & Thiemann, Petra, 2016. "Doing it twice, getting it right? The effects of grade retention and course repetition in higher education," Economics of Education Review, Elsevier, vol. 55(C), pages 198-219.
    10. Maria Paola & Vincenzo Scoppa, 2014. "The effectiveness of remedial courses in Italy: a fuzzy regression discontinuity design," Journal of Population Economics, Springer;European Society for Population Economics, vol. 27(2), pages 365-386, April.
    11. Daniel M. Leeds & Isaac McFarlin & Lindsay Daugherty, 2017. "Does Student Effort Respond to Incentives? Evidence from a Guaranteed College Admissions Program," Research in Higher Education, Springer;Association for Institutional Research, vol. 58(3), pages 231-243, May.
    12. Lindsay Daugherty & Russell Gerber & Francisco Martorell & Trey Miller & Emily Weisburst, 2021. "Heterogeneity in the Effects of College Course Placement," Research in Higher Education, Springer;Association for Institutional Research, vol. 62(7), pages 1086-1111, November.
    13. Qiang Feng & Ming Li & Guangrong Ma, 2018. "Class Tracks And Education Outcomes: Evidence From A Chinese University," Contemporary Economic Policy, Western Economic Association International, vol. 36(2), pages 255-262, April.
    14. Judith Scott-Clayton & Olga Rodriguez, 2014. "Development, Discouragement, or Diversion? New Evidence on the Effects of College Remediation Policy," Education Finance and Policy, MIT Press, vol. 10(1), pages 4-45, November.
    15. Pugatch, Todd & Wilson, Nicholas, 2018. "Nudging study habits: A field experiment on peer tutoring in higher education," Economics of Education Review, Elsevier, vol. 62(C), pages 151-161.
    16. Chen, Yi & Zhao, Yi, 2022. "The timing of first marriage and subsequent life outcomes: Evidence from a natural experiment," Journal of Comparative Economics, Elsevier, vol. 50(3), pages 713-731.
    17. Marcel Fafchamps & Julien Labonne, 2017. "Do Politicians’ Relatives Get Better Jobs? Evidence from Municipal Elections," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 33(2), pages 268-300.
    18. Fanny Landaud & Son Thierry Ly & Éric Maurin, 2020. "Competitive Schools and the Gender Gap in the Choice of Field of Study," Journal of Human Resources, University of Wisconsin Press, vol. 55(1), pages 278-308.
    19. Ivan A Canay & Vishal Kamat, 2018. "Approximate Permutation Tests and Induced Order Statistics in the Regression Discontinuity Design," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 85(3), pages 1577-1608.
    20. Maria Alice Moz-Christofoletti & Paula Carvalho Pereda & Wesley Campanharo, 2022. "Does Decentralized and Voluntary Commitment Reduce Deforestation? The Effects of Programa Municípios Verdes," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 82(1), pages 65-100, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:37:y:2013:i:5:p:370-404. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.