IDEAS home Printed from https://ideas.repec.org/a/gam/jijerp/v17y2020i9p3249-d354728.html
   My bibliography  Save this article

Evaluating the Implementation Quality of a Social and Emotional Learning Program: A Mixed Methods Approach

Author

Listed:
  • Katherine Dowling

    (World Health Organization Collaborating Centre for Health Promotion, National University of Ireland Galway, University Road, H91TK33 Galway, Ireland)

  • Margaret M. Barry

    (World Health Organization Collaborating Centre for Health Promotion, National University of Ireland Galway, University Road, H91TK33 Galway, Ireland)

Abstract

School-based social and emotional learning (SEL) programs have been shown to be effective in producing positive outcomes for adolescents. However, variability in implementation quality can have a negative impact on these program effects. The aim of this current study is to examine the variability in implementation quality for schools implementing the MindOut program and to identify factors that were likely to contribute to this variability. Employing a mixed methods approach, quantitative and qualitative implementation data were collected from teachers (n = 16) and students (n = 280) who participated in the MindOut program. Quantitative indicators were used to score schools’ implementation quality across four dimensions (dosage, adherence/fidelity, quality of delivery and participant responsiveness), and these were averaged to determine overall level of implementation (high/low). Qualitative data identified factors that contributed to implementation quality, and factors were then analyzed in accordance with the schools’ implementation level grouping. Findings indicated that variability in implementation quality existed both between and within schools. A total of eight schools were assigned as high implementers and another eight as low implementers. Influencing factors were categorized into five themes: (i) program factors, (ii) participant factors, (iii) teacher factors, (iv) school contextual factors, and (v) organizational capacity factors. Several differences between high and low implementers were found in relation to these influencing factors. The findings contribute to the evidence on implementation quality in schools by advancing knowledge on measuring implementation quality across multiple dimensions and informants successfully. These findings can also inform practitioners of the main influencing factors in schools so that strategies can be developed to optimize implementation quality in the future.

Suggested Citation

  • Katherine Dowling & Margaret M. Barry, 2020. "Evaluating the Implementation Quality of a Social and Emotional Learning Program: A Mixed Methods Approach," IJERPH, MDPI, vol. 17(9), pages 1-17, May.
    • Handle: RePEc:gam:jijerp:v:17:y:2020:i:9:p:3249-:d:354728
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1660-4601/17/9/3249/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1660-4601/17/9/3249/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Saunders, Ruth P. & Ward, Dianne & Felton, Gwen M. & Dowda, Marsha & Pate, Russell R., 2006. "Examining the link between program implementation and behavior outcomes in the lifestyle education for activity program (LEAP)," Evaluation and Program Planning, Elsevier, vol. 29(4), pages 352-364, November.
    2. Spoth, R. & Trudeau, L. & Shin, C. & Ralston, E. & Redmond, C. & Greenberg, M. & Feinberg, M., 2013. "Longitudinal effects of universal preventive intervention on prescription drug misuse: Three randomized controlled trials with late adolescents and young adults," American Journal of Public Health, American Public Health Association, vol. 103(4), pages 665-672.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Angélica Ferreira Melo & Woska Pires da Costa & Rhanya Rafaella Rodrigues & Lorena de Almeida Cavalcante Brandão Nunes & Priscilla Rayanne E Silva Noll & Matias Noll, 2023. "Panorama of Undergraduate Research in Brazil: Profile, Scientific Production, and Perceptions," Publications, MDPI, vol. 11(2), pages 1-24, May.
    2. Ana Xavier & Paula Vagos & Lara Palmeira & Paulo Menezes & Bruno Patrão & Sónia Pereira & Vanessa Rocha & Sofia Mendes & Marta Tavares, 2022. "Children’s Perspectives on Using Serious Games as a Complement to Promoting Their Social–Emotional Skills," IJERPH, MDPI, vol. 19(15), pages 1-10, August.
    3. Rebecca Bailey & Natasha Raisch & Sonya Temko & Britt Titus & Jonah Bautista & Tahirat Omolara Eniola & Stephanie M. Jones, 2021. "Innovations in Social and Emotional Learning Research and Practice: Building from Evidence and Applying Behavioral Insights to the Design of a Social and Emotional Learning Intervention in Northeast N," IJERPH, MDPI, vol. 18(14), pages 1-21, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alia, Kassandra A. & Wilson, Dawn K. & McDaniel, Tyler & St. George, Sara M. & Kitzman-Ulrich, Heather & Smith, Kelsey & Heatley, VaShawn & Wise, Courtney, 2015. "Development of an innovative process evaluation approach for the Families Improving Together (FIT) for weight loss trial in African American adolescents," Evaluation and Program Planning, Elsevier, vol. 49(C), pages 106-116.
    2. Sara Hoy & Björg Helgadóttir & Åsa Norman, 2022. "Quantitative Measurements for Factors Influencing Implementation in School Settings: Protocol for A Systematic Review and A Psychometric and Pragmatic Analysis," IJERPH, MDPI, vol. 19(19), pages 1-11, October.
    3. Huang, Chien-Chung & Chen, Yafan & Greene, Loren & Cheung, Shannon & Wei, Yuhan, 2019. "Resilience and emotional and behavioral problems of adolescents in China: Effects of a short-term and intensive mindfulness and life skills training," Children and Youth Services Review, Elsevier, vol. 100(C), pages 291-297.
    4. Marieke De Craemer & Vera Verbestel & Maïté Verloigne & Odysseas Androutsos & Luis Moreno & Violeta Iotova & Berthold Koletzko & Piotr Socha & Yannis Manios & Greet Cardon, 2020. "Combining Effect and Process Evaluation on European Preschool Children’s Snacking Behavior in a Kindergarten-Based, Family-Involved Cluster Randomized Controlled Trial: The ToyBox Study," IJERPH, MDPI, vol. 17(19), pages 1-20, October.
    5. Dominick, Gregory M. & Saunders, Ruth P. & Dowda, Marsha & Kenison, Kelli & Evans, Alexandra E., 2014. "Effects of a structural intervention and implementation on physical activity among youth in residential children's homes," Evaluation and Program Planning, Elsevier, vol. 46(C), pages 72-79.
    6. Legrand, Karine & Minary, Laetitia & Briançon, Serge, 2018. "Exploration of the experiences, practices and needs of health promotion professionals when evaluating their interventions and programmes," Evaluation and Program Planning, Elsevier, vol. 70(C), pages 67-72.
    7. Saunders, Ruth P. & Wilcox, Sara & Baruth, Meghan & Dowda, Marsha, 2014. "Process evaluation methods, implementation fidelity results and relationship to physical activity and healthy eating in the Faith, Activity, and Nutrition (FAN) study," Evaluation and Program Planning, Elsevier, vol. 43(C), pages 93-102.
    8. Russell R. Pate & Rod K. Dishman & Marsha Dowda & Kerry L. McIver & Karin A. Pfeiffer & Dwayne E. Porter & Ruth P. Saunders & Dianne S. Ward, 2022. "A Summary of One Research Team’s Contributions to Understanding Physical Activity Behavior in Children and Youth," IJERPH, MDPI, vol. 19(21), pages 1-23, October.
    9. Tran, Van Dinh & Jancey, Jonine & Lee, Andy & James, Anthony & Howat, Peter & Thi Phuong Mai, Le, 2017. "Physical activity and nutrition program for adults with metabolic syndrome: Process evaluation," Evaluation and Program Planning, Elsevier, vol. 61(C), pages 128-133.
    10. Gorman, Dennis M., 2017. "The decline effect in evaluations of the impact of the Strengthening Families Program for Youth 10-14 (SFP 10-14) on adolescent substance use," Children and Youth Services Review, Elsevier, vol. 81(C), pages 29-39.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jijerp:v:17:y:2020:i:9:p:3249-:d:354728. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.