IDEAS home Printed from https://ideas.repec.org/p/feb/natura/00741.html
   My bibliography  Save this paper

The role of open science practices in scaling evidence-based prevention programs

Author

Listed:
  • Robert Ammerman
  • Anne Duggan
  • John List
  • Lauren Supplee
  • Dana Suskind

Abstract

The goal of creating evidence-based programs is to scale them at sufficient breadth to support population-level improvements in critical outcomes. However, this promise is challenging to fulfill. One of the biggest issues for the field is the reduction in effect sizes seen when a program is taken to scale. This paper discusses an economic perspective that identifies the underlying incentives in the research process that lead to scale up problems and to deliver potential solutions to strengthen outcomes at scale. The principles of open science are well aligned with this goal. One prevention program that has begun to scale across the United States is early childhood home visiting. While there is a substantial impact research on home visiting, overall average effect size is .10 and a recent national randomized trial found attenuated effect sizes in programs implemented under real-world conditions. The paper concludes with a case study of the relevance of the economic model and open science in developing and scaling evidence-based home visiting. The case study considers how the traditional approach for testing interventions has influenced home visiting's evolution to date and how open science practices could have supported efforts to maintain impacts while scaling home visiting. It concludes by considering how open science can accelerate the refinement and scaling of home visiting interventions going forward, through accelerated translation of research into policy and practice.

Suggested Citation

  • Robert Ammerman & Anne Duggan & John List & Lauren Supplee & Dana Suskind, 2021. "The role of open science practices in scaling evidence-based prevention programs," Natural Field Experiments 00741, The Field Experiments Website.
  • Handle: RePEc:feb:natura:00741
    as

    Download full text from publisher

    File URL: http://s3.amazonaws.com/fieldexperiments-papers2/papers/00741.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Abhijit Banerjee & Rukmini Banerji & James Berry & Esther Duflo & Harini Kannan & Shobhini Mukerji & Marc Shotland & Michael Walton, 2017. "From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 73-102, Fall.
    2. Bold, Tessa & Kimenyi, Mwangi & Mwabu, Germano & Ng’ang’a, Alice & Sandefur, Justin, 2018. "Experimental evidence on scaling up education reforms in Kenya," Journal of Public Economics, Elsevier, vol. 168(C), pages 1-20.
    3. Michael J. Weiss & Howard S. Bloom & Thomas Brock, 2014. "A Conceptual Framework For Studying The Sources Of Variation In Program Effects," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 33(3), pages 778-808, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Agostinelli, Francesco & Avitabile, Ciro & Bobba, Matteo, 2021. "Enhancing Human Capital at Scale," TSE Working Papers 21-1196, Toulouse School of Economics (TSE).
    2. Gonzales Mariella & Gianmarco León-Ciliotta & Luis R. Martinez, 2018. "How effective are monetary incentives to vote? Evidence from a nationwide policy," Economics Working Papers 1667, Department of Economics and Business, Universitat Pompeu Fabra, revised Jul 2019.
    3. Abhijit Banerjee & Esther Duflo & Clément Imbert & Santhosh Mathew & Rohini Pande, 2020. "E-governance, Accountability, and Leakage in Public Programs: Experimental Evidence from a Financial Management Reform in India," American Economic Journal: Applied Economics, American Economic Association, vol. 12(4), pages 39-72, October.
    4. Jason T. Kerwin & Rebecca L. Thornton, 2021. "Making the Grade: The Sensitivity of Education Program Effectiveness to Input Choices and Outcome Measures," The Review of Economics and Statistics, MIT Press, vol. 103(2), pages 251-264, May.
    5. Eduard Marinov, 2019. "The 2019 Nobel Prize in Economics," Economic Thought journal, Bulgarian Academy of Sciences - Economic Research Institute, issue 6, pages 78-116.
    6. Bird, Kelli A. & Castleman, Benjamin L. & Denning, Jeffrey T. & Goodman, Joshua & Lamberton, Cait & Rosinger, Kelly Ochs, 2021. "Nudging at scale: Experimental evidence from FAFSA completion campaigns," Journal of Economic Behavior & Organization, Elsevier, vol. 183(C), pages 105-128.
    7. Omar Al-Ubaydli & John List & Dana Suskind, 2019. "The science of using science: Towards an understanding of the threats to scaling experiments," Artefactual Field Experiments 00670, The Field Experiments Website.
    8. Jules Gazeaud & Claire Ricard, 2021. "Conditional cash transfers and the learning crisis: evidence from Tayssir scale-up in Morocco," NOVAFRICA Working Paper Series wp2102, Universidade Nova de Lisboa, Nova School of Business and Economics, NOVAFRICA.
    9. Fazzio, Ila & Eble, Alex & Lumsdaine, Robin L. & Boone, Peter & Bouy, Baboucarr & Hsieh, Pei-Tseng Jenny & Jayanty, Chitra & Johnson, Simon & Silva, Ana Filipa, 2021. "Large learning gains in pockets of extreme poverty: Experimental evidence from Guinea Bissau," Journal of Public Economics, Elsevier, vol. 199(C).
    10. Agostinelli, Francesco & Avitabile, Ciro & Bobba, Matteo, 2021. "Enhancing Human Capital at Scale," TSE Working Papers 21-1196, Toulouse School of Economics (TSE).
    11. Committee, Nobel Prize, 2019. "Understanding development and poverty alleviation," Nobel Prize in Economics documents 2019-2, Nobel Prize Committee.
    12. Eble, Alex & Frost, Chris & Camara, Alpha & Bouy, Baboucarr & Bah, Momodou & Sivaraman, Maitri & Hsieh, Pei-Tseng Jenny & Jayanty, Chitra & Brady, Tony & Gawron, Piotr & Vansteelandt, Stijn & Boone, P, 2021. "How much can we remedy very low learning levels in rural parts of low-income countries? Impact and generalizability of a multi-pronged para-teacher intervention from a cluster-randomized trial in the ," Journal of Development Economics, Elsevier, vol. 148(C).
    13. Omar Al-Ubaydli & John List & Claire Mackevicius & Min Sok Lee & Dana Suskind, 2019. "How Can Experiments Play a Greater Role in Public Policy? 12 Proposals from an Economic Model of Scaling," Artefactual Field Experiments 00679, The Field Experiments Website.
    14. Blimpo, Moussa P. & Pugatch, Todd, 2021. "Entrepreneurship education and teacher training in Rwanda," Journal of Development Economics, Elsevier, vol. 149(C).
    15. Sabrin A. Beg & Adrienne M. Lucas & Waqas Halim & Umar Saif, 2019. "Engaging Teachers with Technology Increased Achievement, Bypassing Teachers Did Not," NBER Working Papers 25704, National Bureau of Economic Research, Inc.
    16. Bellés Obrero, Cristina & Lombardi, María, 2019. "Teacher Performance Pay and Student Learning: Evidence from a Nationwide Program in Peru," IZA Discussion Papers 12600, Institute of Labor Economics (IZA).
    17. Gilligan, Daniel O. & Karachiwalla, Naureen & Kasirye, Ibrahim & Lucas, Adrienne & Neal, Derek, 2018. "Educator Incentives and Educational Triage in Rural Primary Schools," IZA Discussion Papers 11516, Institute of Labor Economics (IZA).
    18. Mo, Di & Bai, Yu & Shi, Yaojiang & Abbey, Cody & Zhang, Linxiu & Rozelle, Scott & Loyalka, Prashant, 2020. "Institutions, implementation, and program effectiveness: Evidence from a randomized evaluation of computer-assisted learning in rural China," Journal of Development Economics, Elsevier, vol. 146(C).
    19. Rodriguez-Segura, Daniel & Campton, Cole & Crouch, Luis & Slade, Timothy S., 2021. "Looking beyond changes in averages in evaluating foundational learning: Some inequality measures," International Journal of Educational Development, Elsevier, vol. 84(C).
    20. Herrera-Almanza, Catalina & Rosales-Rueda, Maria F., 2020. "Reducing the Cost of Remoteness: Community-Based Health Interventions and Fertility Choices," Journal of Health Economics, Elsevier, vol. 73(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:feb:natura:00741. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.fieldexperiments.com .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joe Seidel The email address of this maintainer does not seem to be valid anymore. Please ask Joe Seidel to update the entry or send us the correct address (email available below). General contact details of provider: http://www.fieldexperiments.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.