IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v37y2013i6p445-489.html
   My bibliography  Save this article

Intraclass Correlations and Covariate Outcome Correlations for Planning Two- and Three-Level Cluster-Randomized Experiments in Education

Author

Listed:
  • Larry V. Hedges
  • E. C. Hedberg

Abstract

Background: Cluster-randomized experiments that assign intact groups such as schools or school districts to treatment conditions are increasingly common in educational research. Such experiments are inherently multilevel designs whose sensitivity (statistical power and precision of estimates) depends on the variance decomposition across levels. This variance decomposition is usually summarized by the intraclass correlation (ICC) structure and, if covariates are used, the effectiveness of the covariates in explaining variation at each level of the design. Objectives: This article provides a compilation of school- and district-level ICC values of academic achievement and related covariate effectiveness based on state longitudinal data systems. These values are designed to be used for planning group-randomized experiments in education. The use of these values to compute statistical power and plan two- and three-level group-randomized experiments is illustrated. Research Design: We fit several hierarchical linear models to state data by grade and subject to estimate ICCs and covariate effectiveness. The total sample size is over 4.8 million students. We then compare our average of state estimates with the national work by Hedges and Hedberg.

Suggested Citation

  • Larry V. Hedges & E. C. Hedberg, 2013. "Intraclass Correlations and Covariate Outcome Correlations for Planning Two- and Three-Level Cluster-Randomized Experiments in Education," Evaluation Review, , vol. 37(6), pages 445-489, December.
    • Handle: RePEc:sae:evarev:v:37:y:2013:i:6:p:445-489
    DOI: 10.1177/0193841X14529126
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X14529126
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X14529126?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nianbo Dong & Benjamin Kelcey & Jessaca Spybrook, 2021. "Design Considerations in Multisite Randomized Trials Probing Moderated Treatment Effects," Journal of Educational and Behavioral Statistics, , vol. 46(5), pages 527-559, October.
    2. Elizabeth Tipton & Robert B. Olsen, "undated". "Enhancing the Generalizability of Impact Studies in Education," Mathematica Policy Research Reports 35d5625333dc480aba9765b3b, Mathematica Policy Research.
    3. Ben Kelcey & Zuchao Shen & Jessaca Spybrook, 2016. "Intraclass Correlation Coefficients for Designing Cluster-Randomized Trials in Sub-Saharan Africa Education," Evaluation Review, , vol. 40(6), pages 500-525, December.
    4. E. C. Hedberg & Larry V. Hedges, 2014. "Reference Values of Within-District Intraclass Correlations of Academic Achievement by District Characteristics," Evaluation Review, , vol. 38(6), pages 546-582, December.
    5. Blair S Grace & Tess Gregory & Luke Collier & Sally Brinkman, 2022. "Clustering of Wellbeing, Engagement and Academic Outcomes in Australian Primary Schools," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 15(6), pages 2171-2195, December.
    6. Larry V. Hedges & Michael Borenstein, 2014. "Conditional Optimal Design in Three- and Four-Level Experiments," Journal of Educational and Behavioral Statistics, , vol. 39(4), pages 257-281, August.
    7. Jessaca Spybrook & Benjamin Kelcey, 2016. "Introduction to Three Special Issues on Design Parameter Values for Planning Cluster Randomized Trials in the Social Sciences," Evaluation Review, , vol. 40(6), pages 491-499, December.
    8. Daniel McNeish & Jeffrey R. Harring & Denis Dumas, 2023. "A multilevel structured latent curve model for disaggregating student and school contributions to learning," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 32(2), pages 545-575, June.
    9. E. C. Hedberg, 2016. "Academic and Behavioral Design Parameters for Cluster Randomized Trials in Kindergarten," Evaluation Review, , vol. 40(4), pages 279-313, August.
    10. Nianbo Dong & Wendy M. Reinke & Keith C. Herman & Catherine P. Bradshaw & Desiree W. Murray, 2016. "Meaningful Effect Sizes, Intraclass Correlations, and Proportions of Variance Explained by Covariates for Planning Two- and Three-Level Cluster Randomized Trials of Social and Behavioral Outcomes," Evaluation Review, , vol. 40(4), pages 334-377, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:37:y:2013:i:6:p:445-489. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.