IDEAS home Printed from https://ideas.repec.org/p/hka/wpaper/2020-039.html
   My bibliography  Save this paper

Implementing Educational Interventions at Scale

Author

Listed:
  • Simon Calmar Andersen

    (Aarhus Universitet)

  • Ulrik Hvidman

Abstract

Although many educational programs have demonstrated the potential to increase student learning, few examples of successful scaling exist. We study the scalability of a parent-aimed reading program that has shown promising results in an experiment within a local government. Using a nationwide experiment among the full population of 2nd-grade children in Danish public schools (n=51,312), we find that the program is less effective at large scale. We provide evidence on potential explanations for the lack of scalability, which suggests that implementation fidelity is the most important barrier to successfully scaling this type of educational interventions.

Suggested Citation

  • Simon Calmar Andersen & Ulrik Hvidman, 2020. "Implementing Educational Interventions at Scale," Working Papers 2020-039, Human Capital and Economic Opportunity Working Group.
    • Handle: RePEc:hka:wpaper:2020-039
    Note: ECI
    as

    Download full text from publisher

    File URL: http://humcap.uchicago.edu/RePEc/hka/wpaper/Andersen_Hvidman_2020_implementing-ed-interventions-at-scale.pdf
    File Function: First version, May 25, 2020
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Bergman, Peter & Edmond-Verley, Chana & Notario-Risk, Nicole, 2018. "Parent skills and information asymmetries: Experimental evidence from home visits and text messages in middle and high schools," Economics of Education Review, Elsevier, vol. 66(C), pages 92-103.
    2. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    3. repec:mpr:mprres:6174 is not listed on IDEAS
    4. Jill Constantine & Daniel Player & Tim Silva & Kristin Hallgren & Mary Grider & John Deke, 2009. "An Evaluation of Teachers Trained Through Different Routes to Certification," Mathematica Policy Research Reports 1a9e2910a01843c2babbc8c64, Mathematica Policy Research.
    5. Kraft, Matthew A. & Rogers, Todd, 2015. "The underutilized potential of teacher-to-parent communication: Evidence from a field experiment," Economics of Education Review, Elsevier, vol. 47(C), pages 49-63.
    6. James J. Heckman, 1991. "Randomization and Social Policy Evaluation Revisited," NBER Technical Working Papers 0107, National Bureau of Economic Research, Inc.
    7. repec:mpr:mprres:6676 is not listed on IDEAS
    8. Peter Bergman, 2021. "Parent-Child Information Frictions and Human Capital Investment: Evidence from a Field Experiment," Journal of Political Economy, University of Chicago Press, vol. 129(1), pages 286-322.
    9. Benjamin N. York & Susanna Loeb & Christopher Doss, 2019. "One Step at a Time: The Effects of an Early Literacy Text-Messaging Program for Parents of Preschoolers," Journal of Human Resources, University of Wisconsin Press, vol. 54(3), pages 537-566.
    10. Samuel Berlinski & Matias Busso & Taryn Dinkelman & Claudia Martínez A., 2021. "Reducing Parent-School Information Gaps and Improving Education Outcomes: Evidence from High-Frequency Text Messages," NBER Working Papers 28581, National Bureau of Economic Research, Inc.
    11. repec:mpr:mprres:7962 is not listed on IDEAS
    12. Peter Leopold S. Bergman & Todd Rogers, 2017. "The Impact of Defaults on Technology Adoption, and its Underappreciation by Policymakers," CESifo Working Paper Series 6721, CESifo.
    13. Omar Al-Ubaydli & John A. List & Dana L. Suskind, 2017. "What Can We Learn from Experiments? Understanding the Threats to the Scalability of Experimental Results," American Economic Review, American Economic Association, vol. 107(5), pages 282-286, May.
    14. repec:mpr:mprres:7890 is not listed on IDEAS
    15. repec:mpr:mprres:6175 is not listed on IDEAS
    16. repec:mpr:mprres:7338 is not listed on IDEAS
    17. Karthik Muralidharan & Paul Niehaus, 2017. "Experimentation at Scale," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 103-124, Fall.
    18. Christopher Doss & Erin M. Fahle & Susanna Loeb & Benjamin N. York, 2019. "More Than Just a Nudge: Supporting Kindergarten Parents with Differentiated and Personalized Text Messages," Journal of Human Resources, University of Wisconsin Press, vol. 54(3), pages 567-603.
    19. repec:mpr:mprres:7293 is not listed on IDEAS
    20. Melissa Clark & Hanley Chiang & Sheena McConnell & Tim Silva & Kathy Sonnenfeld & Anastasia Erbe & Mike Puma, "undated". "The Effectiveness of Secondary Math Teachers from Teach For America and the Teaching Fellows Programs (Presentation)," Mathematica Policy Research Reports 32d4ed3f97b14a208a78435db, Mathematica Policy Research.
    21. repec:mpr:mprres:6585 is not listed on IDEAS
    22. Jill Constantine, 2009. "Evaluation of Teachers Trained Through Different Routes to Certification," Mathematica Policy Research Reports 6e759a791c77408a816add035, Mathematica Policy Research.
    23. repec:mpr:mprres:6720 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Peter Bergman, 2020. "Nudging Technology Use: Descriptive and Experimental Evidence from School Information Systems," Education Finance and Policy, MIT Press, vol. 15(4), pages 623-647, Fall.
    2. Noam Angrist & Peter Bergman & Moitshepi Matsheng, 2022. "Experimental evidence on learning using low-tech when school is out," Nature Human Behaviour, Nature, vol. 6(7), pages 941-950, July.
    3. Maldonado, Joana Elisa & De Witte, Kristof, 2021. "The impact of information provision to parents: Experimental evidence on student outcomes," Journal of Behavioral and Experimental Finance, Elsevier, vol. 31(C).
    4. Esposito Acosta,Bruno Nicola & Sautmann,Anja, 2022. "Adaptive Experiments for Policy Choice : Phone Calls for Home Reading in Kenya," Policy Research Working Paper Series 10098, The World Bank.
    5. Damgaard, Mette Trier & Nielsen, Helena Skyt, 2018. "Nudging in education," Economics of Education Review, Elsevier, vol. 64(C), pages 313-342.
    6. Joana Elisa Maldonado & Kristof De Witte & Koen Declercq, 2022. "The effects of parental involvement in homework: two randomised controlled trials in financial education," Empirical Economics, Springer, vol. 62(3), pages 1439-1464, March.
    7. Hanley S. Chiang & Melissa A. Clark & Sheena McConnell, "undated". "Supplying Disadvantaged Schools with Effective Teachers: Experimental Evidence on Secondary Math Teachers from Teach For America," Mathematica Policy Research Reports 4623b3b72e154403be21e3878, Mathematica Policy Research.
    8. Justus J. Randolph & Anaya Bryson & Lakshmi Menon & Stephen Michaels & Debra Leigh Walls Rosenstein & Warren McPherson, 2016. "PROTOCOL: Montessori Education for Improving Academic and Social/Behavioral Outcomes for Elementary Students," Campbell Systematic Reviews, John Wiley & Sons, vol. 12(1), pages 1-32.
    9. Roland G. Fryer, Jr, 2016. "The Production of Human Capital in Developed Countries: Evidence from 196 Randomized Field Experiments," NBER Working Papers 22130, National Bureau of Economic Research, Inc.
    10. Hermes, Henning & Lergetporer, Philipp & Peter, Frauke & Wiederhold, Simon, 2021. "Behavioral Barriers and the Socioeconomic Gap in Child Care Enrollment," IZA Discussion Papers 14698, Institute of Labor Economics (IZA).
    11. Lisa Dragoset & Susanne James-Burdumy & Kristin Hallgren & Irma Perez-Johnson & Mariesa Herrmann & Christina Tuttle & Megan Hague Angus & Rebecca Herman & Matthew Murray & Courtney Tanenbaum & Cheryl , 2015. "Usage of Practices Promoted by School Improvement Grants," Mathematica Policy Research Reports 8e99f01663504ef5b9f8357f6, Mathematica Policy Research.
    12. Henning Hermes & Philipp Lergetporer & Frauke Peter & Simon Wiederhold, 2021. "Application Barriers and the Socioeconomic Gap in Child Care Enrollment," CESifo Working Paper Series 9282, CESifo.
    13. Cortes, Kalena E. & Fricke, Hans & Loeb, Susanna & Song, David S. & York, Benjamin N., 2023. "When behavioral barriers are too high or low – How timing matters for text-based parenting interventions," Economics of Education Review, Elsevier, vol. 92(C).
    14. Noam Angrist & Peter Bergman & Moitshepi Matsheng, 2020. "School’s Out: Experimental Evidence on Limiting Learning Loss Using “Low-Tech” in a Pandemic," NBER Working Papers 28205, National Bureau of Economic Research, Inc.
    15. Lisa Dragoset & Jaime Thomas & Mariesa Herrmann & John Deke & Susanne James-Burdumy & Cheryl Graczewski & Andrea Boyle & Rachel Upton & Courtney Tanenbaum & Jessica Giffin, "undated". "School Improvement Grants: Implementation and Effectiveness (Final Report)," Mathematica Policy Research Reports 76bce3f4bb0944f29a481fae0, Mathematica Policy Research.
    16. Amaral, Sofia & Dinarte-Diaz, Lelys & Dominguez, Patricio & Perez-Vincent, Santiago M., 2024. "Helping families help themselves: The (Un)intended impacts of a digital parenting program," Journal of Development Economics, Elsevier, vol. 166(C).
    17. Lisa Dragoset & Jaime Thomas & Mariesa Herrmann & John Deke & Susanne James-Burdumy & Cheryl Graczewski & Andrea Boyle & Courtney Tanenbaum & Jessica Giffin & Rachel Upton, "undated". "Race to the Top: Implementation and Relationship to Student Outcomes," Mathematica Policy Research Reports 4169441544e144b08777d967e, Mathematica Policy Research.
    18. Lisa Dragoset & Susanne James-Burdumy & Kristin Hallgren & Irma Perez-Johnson & Mariesa Herrmann & Christina Tuttle & Megan Hague Angus & Rebecca Herman & Matthew Murray & Corutney Tanenbaum & Cheryl , 2015. "Usage of Policies and Practices Promoted by Race to the Top," Mathematica Policy Research Reports 4b296d4310f843acad1f80b5a, Mathematica Policy Research.
    19. Tim Silva & Allison McKie & Virginia Knechtel & Philip Gleason & Libby Makowsky, "undated". "Teaching Residency Programs: A Multisite Look at a New Model to Prepare Teachers for High-Need Schools," Mathematica Policy Research Reports 58dae816db6b4d008b73ac09d, Mathematica Policy Research.
    20. Tim Kautz & Peter Z. Schochet & Charles Tilley, "undated". "Comparing Impact Findings from Design-Based and Model-Based Methods: An Empirical Investigation," Mathematica Policy Research Reports b7656ddce20f4007b71836e99, Mathematica Policy Research.

    More about this item

    Keywords

    implementation; experiment;

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hka:wpaper:2020-039. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Jennifer Pachon (email available below). General contact details of provider: https://edirc.repec.org/data/mfichus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.