IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v50y2025i1p44-71.html

Power Analyses for Estimation of Complier Average Causal Effects Under Random Encouragement Designs in Education Research: Theory and Guidance

Author

Listed:
  • Peter Z. Schochet

Abstract

Random encouragement designs evaluate treatments that aim to increase participation in a program or activity. These randomized controlled trials (RCTs) can also assess the mediated effects of participation itself on longer term outcomes using a complier average causal effect (CACE) estimation framework. This article considers power analysis methods for such CACE analyses for a range of RCT designs, including nonclustered, clustered, and random block designs. The focus is on behavioral encouragements to promote action, such as text messaging, that are increasingly being tested in education trials. We derive asymptotic distributions of the CACE estimators using generalized estimating equations theory, which underlie the power formulas. We incorporate noncompliance from both the actual receipt of the encouragement and participation itself. An illustrative power analysis provides sample size guidance using an available Shiny R dashboard.

Suggested Citation

  • Peter Z. Schochet, 2025. "Power Analyses for Estimation of Complier Average Causal Effects Under Random Encouragement Designs in Education Research: Theory and Guidance," Journal of Educational and Behavioral Statistics, , vol. 50(1), pages 44-71, February.
    • Handle: RePEc:sae:jedbes:v:50:y:2025:i:1:p:44-71
    DOI: 10.3102/10769986241233790
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/10769986241233790
    Download Restriction: no
    File URL: https://libkey.io/10.3102/10769986241233790?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Hummel, Dennis & Maedche, Alexander, 2019. "How effective is nudging? A quantitative review on the effect sizes and limits of empirical nudging studies," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 80(C), pages 47-58.
    2. Kellie Ottoboni & Jason Poulos, 2019. "Estimating population average treatment effects from experiments with noncompliance," Papers 1901.02991, arXiv.org, revised Aug 2020.
    3. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    4. Peter Bergman & Eric W. Chan, 2021. "Leveraging Parents through Low-Cost Technology: The Impact of High-Frequency Information on Student Achievement," Journal of Human Resources, University of Wisconsin Press, vol. 56(1), pages 125-158.
    5. Xuran Wang & Yang Jiang & Nancy R. Zhang & Dylan S. Small, 2018. "Sensitivity analysis and power for instrumental variable studies," Biometrics, The International Biometric Society, vol. 74(4), pages 1150-1160, December.
    6. repec:mpr:mprres:5863 is not listed on IDEAS
    7. Abadie A., 2002. "Bootstrap Tests for Distributional Treatment Effects in Instrumental Variable Models," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 284-292, March.
    8. Kosuke Imai & Dustin Tingley & Teppei Yamamoto, 2013. "Experimental designs for identifying causal mechanisms," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(1), pages 5-51, January.
    9. Bird, Kelli A. & Castleman, Benjamin L. & Denning, Jeffrey T. & Goodman, Joshua & Lamberton, Cait & Rosinger, Kelly Ochs, 2021. "Nudging at scale: Experimental evidence from FAFSA completion campaigns," Journal of Economic Behavior & Organization, Elsevier, vol. 183(C), pages 105-128.
    10. A. Colin Cameron & Douglas L. Miller, 2015. "A Practitioner’s Guide to Cluster-Robust Inference," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 317-372.
    11. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, Enero-Abr.
    12. repec:mpr:mprres:7090 is not listed on IDEAS
    13. repec:mpr:mprres:7071 is not listed on IDEAS
    14. Hansen, Bruce E. & Lee, Seojeong, 2019. "Asymptotic theory for clustered samples," Journal of Econometrics, Elsevier, vol. 210(2), pages 268-290.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Markus Frölich & Martin Huber, 2017. "Direct and indirect treatment effects–causal chains and mediation analysis with instrumental variables," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(5), pages 1645-1666, November.
    2. A. Belloni & V. Chernozhukov & I. Fernández‐Val & C. Hansen, 2017. "Program Evaluation and Causal Inference With High‐Dimensional Data," Econometrica, Econometric Society, vol. 85, pages 233-298, January.
    3. Atila Abdulkadiroğlu & Joshua D. Angrist & Yusuke Narita & Parag A. Pathak, 2017. "Research Design Meets Market Design: Using Centralized Assignment for Impact Evaluation," Econometrica, Econometric Society, vol. 85, pages 1373-1432, September.
    4. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    5. Yuta Ota & Takahiro Hoshino & Taisuke Otsu, 2024. "Causal Inference With Auxiliary Observations," Keio-IES Discussion Paper Series 2024-022, Institute for Economics Studies, Keio University.
    6. MacKinnon, James G. & Nielsen, Morten Ørregaard & Webb, Matthew D., 2023. "Cluster-robust inference: A guide to empirical practice," Journal of Econometrics, Elsevier, vol. 232(2), pages 272-299.
    7. Ottoboni Kellie N. & Poulos Jason V., 2020. "Estimating population average treatment effects from experiments with noncompliance," Journal of Causal Inference, De Gruyter, vol. 8(1), pages 108-130, January.
    8. Sun, Zhenting, 2023. "Instrument validity for heterogeneous causal effects," Journal of Econometrics, Elsevier, vol. 237(2).
    9. Blaise Melly und Kaspar W thrich, 2016. "Local quantile treatment effects," Diskussionsschriften dp1605, Universitaet Bern, Departement Volkswirtschaft.
    10. Undral Byambadalai & Tomu Hirata & Tatsushi Oka & Shota Yasui, 2025. "Beyond the Average: Distributional Causal Inference under Imperfect Compliance," Papers 2509.15594, arXiv.org, revised Oct 2025.
    11. Grenet, Julien & Grönqvist, Hans & Niknami, Susan, 2024. "The effects of electronic monitoring on offenders and their families," Journal of Public Economics, Elsevier, vol. 230(C).
    12. James G. MacKinnon & Matthew D. Webb, 2020. "When and How to Deal with Clustered Errors in Regression Models," Working Paper 1421, Economics Department, Queen's University.
    13. Andrés Elberg & Pedro M. Gardete & Rosario Macera & Carlos Noton, 2019. "Dynamic effects of price promotions: field evidence, consumer search, and supply-side implications," Quantitative Marketing and Economics (QME), Springer, vol. 17(1), pages 1-58, March.
    14. Pedro H. C. Sant'Anna & Xiaojun Song & Qi Xu, 2022. "Covariate distribution balance via propensity scores," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(6), pages 1093-1120, September.
    15. Federico Belotti & Edoardo Di Porto & Gianluca Santoni, 2021. "The effect of local taxes on firm performance: Evidence from geo‐referenced data," Journal of Regional Science, Wiley Blackwell, vol. 61(2), pages 492-510, March.
    16. Undral Byambadalai & Tatsushi Oka & Shota Yasui, 2024. "Estimating Distributional Treatment Effects in Randomized Experiments: Machine Learning for Variance Reduction," Papers 2407.16037, arXiv.org.
    17. Joshua D. Angrist & Sarah R. Cohodes & Susan M. Dynarski & Parag A. Pathak & Christopher R. Walters, 2016. "Stand and Deliver: Effects of Boston's Charter High Schools on College Preparation, Entry, and Choice," Journal of Labor Economics, University of Chicago Press, vol. 34(2), pages 275-318.
    18. Victor Chernozhukov & Ivan Fernandez-Val & Christian Hansen, 2013. "Program evaluation with high-dimensional data," CeMMAP working papers CWP57/13, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    19. Wenjie Wang & Yichong Zhang, 2021. "Wild Bootstrap for Instrumental Variables Regressions with Weak and Few Clusters," Papers 2108.13707, arXiv.org, revised Jan 2024.
    20. Timothy Gubler & Ian Larkin & Lamar Pierce, 2018. "Doing Well by Making Well: The Impact of Corporate Wellness Programs on Employee Productivity," Management Science, INFORMS, vol. 64(11), pages 4967-4987, November.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:50:y:2025:i:1:p:44-71. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.