IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp2412.html
   My bibliography  Save this paper

The Power of the Test in Three-Level Designs

Author

Listed:
  • Konstantopoulos, Spyros

    (Michigan State University)

Abstract

Field experiments that involve nested structures may assign treatment conditions either to entire groups (such as classrooms or schools), or individuals within groups (such as students). Since field experiments involve clustering, key aspects of their design include knowledge of the intraclass correlation structure and the sample sizes necessary to achieve adequate power to detect the treatment effect. This study provides methods for computing power in three-level designs, where for example, students are nested within classrooms and classrooms are nested within schools. The power computations take into account clustering effects at the classroom and at the school level, sample size effects (e.g., number of students, classrooms, and schools), and covariate effects (e.g., pre-treatment measures). The methods are generalizable to quasi-experimental studies that examine group differences in an outcome, or associations between predictors and outcomes.

Suggested Citation

  • Konstantopoulos, Spyros, 2006. "The Power of the Test in Three-Level Designs," IZA Discussion Papers 2412, Institute of Labor Economics (IZA).
  • Handle: RePEc:iza:izadps:dp2412
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp2412.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Murray, D.M. & Varnell, S.P. & Blitstein, J.L., 2004. "Design and Analysis of Group-Randomized Trials: A Review of Recent Methodological Developments," American Journal of Public Health, American Public Health Association, vol. 94(3), pages 423-432.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jonathan L. Blitstein & David M. Murray & Peter J. Hannan & William R. Shadish, 2005. "Increasing the Degrees of Freedom in Future Group Randomized Trials," Evaluation Review, , vol. 29(3), pages 268-286, June.
    2. Jonathan L. Blitstein & Peter J. Hannan & David M. Murray & William R. Shadish, 2005. "Increasing the Degrees of Freedom in Existing Group Randomized Trials," Evaluation Review, , vol. 29(3), pages 241-267, June.
    3. Ji-Hyun Lee & Michael J Schell & Richard Roetzheim, 2009. "Analysis of Group Randomized Trials with Multiple Binary Endpoints and Small Number of Groups," PLOS ONE, Public Library of Science, vol. 4(10), pages 1-9, October.
    4. Siying Guo & Jianxuan Liu & Qiu Wang, 2022. "Effective Learning During COVID-19: Multilevel Covariates Matching and Propensity Score Matching," Annals of Data Science, Springer, vol. 9(5), pages 967-982, October.
    5. repec:mpr:mprres:4632 is not listed on IDEAS
    6. Peter Z. Schochet, "undated". "Statistical Power for Random Assignment Evaluations of Education Programs," Mathematica Policy Research Reports 6749d31ad72d4acf988f7dce5, Mathematica Policy Research.
    7. Benjamin F. Arnold & Francois Rerolle & Christine Tedijanto & Sammy M. Njenga & Mahbubur Rahman & Ayse Ercumen & Andrew Mertens & Amy J. Pickering & Audrie Lin & Charles D. Arnold & Kishor Das & Chris, 2024. "Geographic pair matching in large-scale cluster randomized trials," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    8. Hélène Sancho-Garnier & Bruno Pereira & Pierre Césarini, 2012. "A Cluster Randomized Trial to Evaluate a Health Education Programme “Living with Sun at School”," IJERPH, MDPI, vol. 9(7), pages 1-17, July.
    9. Yasaman Jamshidi-Naeini & Armando Pena & Abu Bakkar Siddique & Andrew W. Brown & David B. Allison, 2023. "One Cluster per Condition Is Not a Valid Design in Cluster-Randomized Trials. Comment on Wang et al. The Effect of Different Physical Exercise Programs on Physical Fitness among Preschool Children: A ," IJERPH, MDPI, vol. 20(17), pages 1-2, August.
    10. Raphiel Murden & Jon Agley & Lilian Golzarri-Arroyo & Armando Peña & Danny Valdez & Abu Bakkar Siddique & Moonseong Heo & David B. Allison, 2023. "Comment on Marsigliante et al. Effects on Children’s Physical and Mental Well-Being of a Physical-Activity-Based School Intervention Program: A Randomized Study. Int. J. Environ. Res. Public Health 20," IJERPH, MDPI, vol. 20(23), pages 1-5, December.
    11. Larry V. Hedges, 2009. "Adjusting a Significance Test for Clustering in Designs With Two Levels of Nesting," Journal of Educational and Behavioral Statistics, , vol. 34(4), pages 464-490, December.
    12. Jose Giovany Babativa-Márquez & José Luis Vicente-Villardón, 2021. "Logistic Biplot by Conjugate Gradient Algorithms and Iterated SVD," Mathematics, MDPI, vol. 9(16), pages 1-19, August.
    13. Spyros Konstantopoulos, 2009. "Incorporating Cost in Power Analysis for Three-Level Cluster-Randomized Designs," Evaluation Review, , vol. 33(4), pages 335-357, August.
    14. Ruoxuan Xiong & Susan Athey & Mohsen Bayati & Guido Imbens, 2019. "Optimal Experimental Design for Staggered Rollouts," Papers 1911.03764, arXiv.org, revised Sep 2023.
    15. Fang Yu & Nicholas A Hein & Danstan S Bagenda, 2020. "Preventing HIV and HSV-2 through knowledge and attitudes: A replication study of a multi-component community-based intervention in Zimbabwe," PLOS ONE, Public Library of Science, vol. 15(1), pages 1-25, January.
    16. Steven Teerenstra & Bing Lu & John S. Preisser & Theo van Achterberg & George F. Borm, 2010. "Sample Size Considerations for GEE Analyses of Three-Level Cluster Randomized Trials," Biometrics, The International Biometric Society, vol. 66(4), pages 1230-1237, December.
    17. Wang Pengyuan & Traskin Mikhail & Small Dylan S., 2013. "Robust Inferences from a Before-and-After Study with Multiple Unaffected Control Groups," Journal of Causal Inference, De Gruyter, vol. 1(2), pages 209-234, June.
    18. Winfried Zinn & Sebastian Sauer & Richard Göllner, 2016. "The German Inpatient Satisfaction Scale," SAGE Open, , vol. 6(2), pages 21582440166, April.
    19. Dateng Li & Jing Cao & Song Zhang, 2020. "Power analysis for cluster randomized trials with multiple binary co‐primary endpoints," Biometrics, The International Biometric Society, vol. 76(4), pages 1064-1074, December.
    20. Walt Stroup & Elizabeth Claassen, 2020. "Pseudo-Likelihood or Quadrature? What We Thought We Knew, What We Think We Know, and What We Are Still Trying to Figure Out," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 25(4), pages 639-656, December.

    More about this item

    Keywords

    experiments; clustering; nested designs; statistical power; randomized trials; treatment effects; random effects;
    All these keywords.

    JEL classification:

    • C9 - Mathematical and Quantitative Methods - - Design of Experiments

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp2412. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.