IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v83y2020ics0149718920301567.html
   My bibliography  Save this article

Interpreting the effectiveness of a summer reading program: The eye of the beholder

Author

Listed:
  • Reed, Deborah K.
  • Aloe, Ariel M.

Abstract

In applying a methods-oriented approach to evaluation, this study interpreted the effectiveness of a summer reading program from three different stakeholder perspectives: practitioners from the school district, the funding agency supporting the program, and the policymakers considering mandating summer school. Archival data were obtained on 2330 students reading below benchmark in Grades 2–5. After propensity score matching participants to peers who did not attend the summer program, the final sample consisted of 630 students. Pre-to-posttest growth models revealed positive effects in Grades 2–4 (standardized slopes of .40–.54), but fifth graders demonstrated negligible improvement (standardized slope of .15). The standardized mean differences of propensity score matched treatment and control group students indicated null effects in all grade levels (d = −.13 to .05). Achieving proficient reading performance also was not attributable to summer school participation. Findings underscore the importance of operationalizing effectiveness in summative evaluation.

Suggested Citation

  • Reed, Deborah K. & Aloe, Ariel M., 2020. "Interpreting the effectiveness of a summer reading program: The eye of the beholder," Evaluation and Program Planning, Elsevier, vol. 83(C).
    • Handle: RePEc:eee:epplan:v:83:y:2020:i:c:s0149718920301567
    DOI: 10.1016/j.evalprogplan.2020.101852
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718920301567
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2020.101852?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Rosenbaum, Paul R., 2010. "Design Sensitivity and Efficiency in Observational Studies," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 692-702.
    2. repec:mpr:mprres:7461 is not listed on IDEAS
    3. Eric A. Hanushek & John F. Kain & Steven G. Rivkin, 1998. "Does Special Education Raise Academic Achievement for Students with Disabilities?," NBER Working Papers 6690, National Bureau of Economic Research, Inc.
    4. Ho, Daniel E. & Imai, Kosuke & King, Gary & Stuart, Elizabeth A., 2007. "Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference," Political Analysis, Cambridge University Press, vol. 15(3), pages 199-236, July.
    5. Scott Simkins & Stuart Allen, 2000. "Pretesting students to improve teaching and learning," International Advances in Economic Research, Springer;International Atlantic Economic Society, vol. 6(1), pages 100-112, February.
    6. Glenn S. Baete & Craig Hochbein, 2014. "Project Proficiency: Assessing the Independent Effects of High School Reform in an Urban District," The Journal of Educational Research, Taylor & Francis Journals, vol. 107(6), pages 493-511, October.
    7. Clark, William C. & van Kerkhoff, Lorrae & Lebel, Louis & Gallopin, Gilberto, 2016. "Crafting Usuable Knowledge for Sustainable Development," Working Paper Series 16-005, Harvard University, John F. Kennedy School of Government.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhexiao Lin & Peng Ding & Fang Han, 2023. "Estimation Based on Nearest Neighbor Matching: From Density Ratio to Average Treatment Effect," Econometrica, Econometric Society, vol. 91(6), pages 2187-2217, November.
    2. Bo Zhang, 2023. "Efficient algorithms for building representative matched pairs with enhanced generalizability," Biometrics, The International Biometric Society, vol. 79(4), pages 3981-3997, December.
    3. Adam C. Sales & Ben B. Hansen & Brian Rowan, 2018. "Rebar: Reinforcing a Matching Estimator With Predictions From High-Dimensional Covariates," Journal of Educational and Behavioral Statistics, , vol. 43(1), pages 3-31, February.
    4. Riccardo D'Alberto & Francesco Pagliacci & Matteo Zavalloni, 2023. "A socioeconomic impact assessment of three Italian national parks," Journal of Regional Science, Wiley Blackwell, vol. 63(1), pages 114-147, January.
    5. España, F. & Arriagada, R. & Melo, O. & Foster, W., 2022. "Forest plantation subsidies: Impact evaluation of the Chilean case," Forest Policy and Economics, Elsevier, vol. 137(C).
    6. Fukui Hideki, 2023. "Evaluating Different Covariate Balancing Methods: A Monte Carlo Simulation," Statistics, Politics and Policy, De Gruyter, vol. 14(2), pages 205-326, June.
    7. Huang, Wei & Li, Fan & Liao, Xiaowei & Hu, Pingping, 2018. "More money, better performance? The effects of student loans and need-based grants in China's higher education," China Economic Review, Elsevier, vol. 51(C), pages 208-227.
    8. Franz R. Hahn & Werner Hölzl & Claudia Kwapil, 2016. "The Credit Channel and the Role of Monetary Policy Before, During and After the Global Financial Crisis. A Micro Data Approach to the Analysis of Bank-firm Relationships," WIFO Studies, WIFO, number 59233, February.
    9. Stefan KIRCHWEGER & Jochen KANTELHARDT & Friedrich LEISCH, 2015. "Impacts of the government-supported investments on the economic farm performance in Austria," Agricultural Economics, Czech Academy of Agricultural Sciences, vol. 61(8), pages 343-355.
    10. Stephen C. Nelson & Geoffrey P. R. Wallace, 2017. "Are IMF lending programs good or bad for democracy?," The Review of International Organizations, Springer, vol. 12(4), pages 523-558, December.
    11. Zhexiao Lin & Peng Ding & Fang Han, 2021. "Estimation based on nearest neighbor matching: from density ratio to average treatment effect," Papers 2112.13506, arXiv.org.
    12. Bo Zhang & Eric J. Tchetgen Tchetgen, 2022. "A semi‐parametric approach to model‐based sensitivity analysis in observational studies," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(S2), pages 668-691, December.
    13. Yu-Chen Kuo & Jia-Huey Lin, 2020. "Picking the lock: how universal healthcare programs influence entrepreneurial activities," Small Business Economics, Springer, vol. 54(1), pages 3-24, January.
    14. Richard Aviles-Lopez & Juan de Dios Luna del Castillo & Miguel Ángel Montero-Alonso, 2023. "Exploratory Matching Model Search Algorithm (EMMSA) for Causal Analysis: Application to the Cardboard Industry," Mathematics, MDPI, vol. 11(21), pages 1-34, October.
    15. Liu, Yan & Chen, Xi & Yan, Zhijun, 2019. "Depression in the House: The Effects of Household Air Pollution from Solid Fuel Use in China," IZA Discussion Papers 12654, Institute of Labor Economics (IZA).
    16. Noemi Kreif & Richard Grieve & Rosalba Radice & Zia Sadique & Roland Ramsahai & Jasjeet S. Sekhon, 2012. "Methods for Estimating Subgroup Effects in Cost-Effectiveness Analyses That Use Observational Data," Medical Decision Making, , vol. 32(6), pages 750-763, November.
    17. Do, Manh Hung & Nguyen, Trung Thanh & Grote, Ulrike, 2023. "Land consolidation, rice production, and agricultural transformation: Evidence from household panel data for Vietnam," Economic Analysis and Policy, Elsevier, vol. 77(C), pages 157-173.
    18. Federico Biagi & Daniele Bondonio & Alberto Martini, 2015. "Counterfactual Impact Evaluation of Enterprise Support Programmes. Evidence from a Decade of Subsidies to Italian Firm," ERSA conference papers ersa15p1619, European Regional Science Association.
    19. Bono, Pierre-Henri & David, Quentin & Desbordes, Rodolphe & Py, Loriane, 2022. "Metro infrastructure and metropolitan attractiveness," Regional Science and Urban Economics, Elsevier, vol. 93(C).
    20. Zichen Deng & Maarten Lindeboom, 2021. "Early-life Famine Exposure, Hunger Recall and Later-life Health," Tinbergen Institute Discussion Papers 21-054/V, Tinbergen Institute.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:83:y:2020:i:c:s0149718920301567. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.