IDEAS home Printed from https://ideas.repec.org/p/mpr/mprres/eedf2eac4d4c4d8e869052c1d4b2d566.html

Multi-Armed RCTs: A Design-Based Framework

Author

Listed:
  • Peter Z. Schochet

Abstract

This report describes how design-based estimators for the two-group design need to be modified for the multi-armed design when comparing pairs of research groups to each other.

Suggested Citation

  • Peter Z. Schochet, "undated". "Multi-Armed RCTs: A Design-Based Framework," Mathematica Policy Research Reports eedf2eac4d4c4d8e869052c1d, Mathematica Policy Research.
    • Handle: RePEc:mpr:mprres:eedf2eac4d4c4d8e869052c1d4b2d566
    as

    Download full text from publisher

    File URL: https://www.mathematica.org/-/media/publications/pdfs/education/2017/multi-armed-rcts.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Donald B. Rubin, 1977. "Assignment to Treatment Group on the Basis of a Covariate," Journal of Educational and Behavioral Statistics, , vol. 2(1), pages 1-26, March.
    2. Peter Z. Schochet, "undated". "An Approach for Addressing the Multiple Testing Problem in Social Policy Impact Evaluations," Mathematica Policy Research Reports dbb4b45215a740819b08ca578, Mathematica Policy Research.
    3. Peter Z. Schochet & Hanley S. Chiang, 2011. "Estimation and Identification of the Complier Average Causal Effect Parameter in Education RCTs," Journal of Educational and Behavioral Statistics, , vol. 36(3), pages 307-345, June.
    4. Howard S. Bloom, 1984. "Accounting for No-Shows in Experimental Evaluation Designs," Evaluation Review, , vol. 8(2), pages 225-246, April.
    5. repec:mpr:mprres:6192 is not listed on IDEAS
    6. Castleman, Benjamin L. & Page, Lindsay C., 2015. "Summer nudging: Can personalized text messages and peer mentor outreach increase college going among low-income high school graduates?," Journal of Economic Behavior & Organization, Elsevier, vol. 115(C), pages 144-160.
    7. Paul A. Games & John F. Howell, 1976. "Pairwise Multiple Comparison Procedures with Unequal N’s and/or Variances: A Monte Carlo Study," Journal of Educational and Behavioral Statistics, , vol. 1(2), pages 113-125, June.
    8. repec:mpr:mprres:7071 is not listed on IDEAS
    9. repec:mpr:mprres:7723 is not listed on IDEAS
    10. repec:mpr:mprres:6574 is not listed on IDEAS
    11. Roberto Agodini & Barbara Harris & Sally Atkins-Burnett & Sheila Heaviside & Timothy Novak & Robert Murphy, "undated". "Achievement Effects of Four Early Elementary School Math Curricula: Findings from First Graders in 39 Schools," Mathematica Policy Research Reports 467194cd2d6f4cbaba7e23745, Mathematica Policy Research.
    12. Qi Long & Roderick J. A. Little & Xihong Lin, 2010. "Estimating causal effects in trials involving multitreatment arms subject to non‐compliance: a Bayesian framework," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 59(3), pages 513-531, May.
    13. Kraft, Matthew A. & Rogers, Todd, 2015. "The underutilized potential of teacher-to-parent communication: Evidence from a field experiment," Economics of Education Review, Elsevier, vol. 47(C), pages 49-63.
    14. Roberto Agodini, 2009. "Achievement Effects of Four Early Elementary School Math Curricula: Findings from First Graders in 39 Schools (Conference Paper)," Mathematica Policy Research Reports a08d661306a843fa89aff4c30, Mathematica Policy Research.
    15. Jing Cheng & Dylan S. Small, 2006. "Bounds on causal effects in three‐arm trials with non‐compliance," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(5), pages 815-836, November.
    16. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, November.
    17. Tan, Zhiqiang, 2010. "Marginal and Nested Structural Models Using Instrumental Variables," Journal of the American Statistical Association, American Statistical Association, vol. 105(489), pages 157-169.
    18. repec:mpr:mprres:7337 is not listed on IDEAS
    19. Peter Z. Schochet, 2009. "An Approach for Addressing the Multiple Testing Problem in Social Policy Impact Evaluations," Evaluation Review, , vol. 33(6), pages 539-567, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dana Rotz & Paul Burkander & Kenneth Fortson & Sheena McConnell & Peter Schochet & Mary Grider & Linda Molinari & Elias Sanchez-Eppler, "undated". "Providing Public Workforce Services to Job Seekers: 15-Month Impact Findings on the WIA Adult and Dislocated Worker Programs (Technical Supplement)," Mathematica Policy Research Reports 1153336bee1c4a969d5341ef4, Mathematica Policy Research.
    2. Peter Z. Schochet, 2020. "The Complier Average Causal Effect Parameter for Multiarmed RCTs," Evaluation Review, , vol. 44(5-6), pages 410-436, October.
    3. Peter Z. Schochet, 2018. "Design-Based Estimators for Average Treatment Effects for Multi-Armed RCTs," Journal of Educational and Behavioral Statistics, , vol. 43(5), pages 568-593, October.
    4. Dana Rotz & Paul Burkander & Mary Grider & Kenneth Fortson & Linda Molinari & Elias Sanchez-Eppler & Lindsay Cattell, "undated". "Providing Public Workforce Services to Job Seekers: 30-Month Impact Findings on the WIA Adult and Dislocated Worker Programs, Technical Supplement," Mathematica Policy Research Reports db04b33db50a4d45824da5a65, Mathematica Policy Research.
    5. repec:mpr:mprres:7031 is not listed on IDEAS
    6. Duncan Chaplin & Arif Mamun & Ali Protik & John Schurrer & Divya Vohra & Kristine Bos & Hannah Burak & Laura Meyer & Anca Dumitrescu & Christopher Ksoll & Thomas Cook, "undated". "Grid Electricity Expansion in Tanzania by MCC: Findings from a Rigorous Impact Evaluation, Final Report," Mathematica Policy Research Reports 144768f69008442e96369195e, Mathematica Policy Research.
    7. Peter Z. Schochet, "undated". "Statistical Theory for the RCT-YES Software: Design-Based Causal Inference for RCTs," Mathematica Policy Research Reports a0c005c003c242308a92c02dc, Mathematica Policy Research.
    8. Emilie Bagby & Kristine Bos & Anca Dumitrescu & Nicholas Ingwersen & Matt Sloan, "undated". "Niger NECS Impact Evaluation Report," Mathematica Policy Research Reports a759ff92090a4c2aba4caefe6, Mathematica Policy Research.
    9. Arif A. Mamun & Lori Timmins & David C. Stapleton, "undated". "Prospects for an Impact Evaluation of Project SEARCH: An Evaluability Assessment," Mathematica Policy Research Reports 29231db872d5434984051a1bd, Mathematica Policy Research.
    10. Jiannan Lu & Peng Ding & Tirthankar Dasgupta, 2018. "Treatment Effects on Ordinal Outcomes: Causal Estimands and Sharp Bounds," Journal of Educational and Behavioral Statistics, , vol. 43(5), pages 540-567, October.
    11. Peter Z. Schochet & Ronald D'Amico & Jillian Berk & Sarah Dolfin & Nathan Wozny, "undated". "Estimated Impacts for Participants in the Trade Adjustment Assistance (TAA) Program Under the 2002 Amendments," Mathematica Policy Research Reports 582d8723f6884d4eb7a3f95a4, Mathematica Policy Research.
    12. Karnani, Mohit, 2016. "Freshmen teachers and college major choice: Evidence from a random assignment in Chile," MPRA Paper 76062, University Library of Munich, Germany.
    13. repec:mpr:mprres:7273 is not listed on IDEAS
    14. Kalena E. Cortes & Hans Fricke & Susanna Loeb & David S. Song, 2018. "Too little or too much? Actionable Advice in an Early-Childhood Text Messaging Experiment," NBER Working Papers 24827, National Bureau of Economic Research, Inc.
    15. John Deke & Lisa Dragoset, "undated". "Statistical Power for Regression Discontinuity Designs in Education: Empirical Estimates of Design Effects Relative to Randomized Controlled Trials," Mathematica Policy Research Reports a4f1d03eb7bf427a8983d4736, Mathematica Policy Research.
    16. repec:mpr:mprres:7909 is not listed on IDEAS
    17. Fricke, Hans & Kalogrides, Demetra & Loeb, Susanna, 2018. "It’s too annoying: Who drops out of educational text messaging programs and why," Economics Letters, Elsevier, vol. 173(C), pages 39-43.
    18. Samuel Berlinski & Matias Busso & Taryn Dinkelman & Claudia Martínez A., 2025. "Reducing Parent–School Information Gaps and Improving Education Outcomes: Evidence from High-Frequency Text Messages," Journal of Human Resources, University of Wisconsin Press, vol. 60(4), pages 1284-1322.
    19. Cory Koedel & Rachana Bhatt, 2011. "Large-Scale Evaluations of Curricular Effectiveness: The Case of Elementary Mathematics in Indiana," Working Papers 1122, Department of Economics, University of Missouri, revised 31 Jan 2012.
    20. Lindsay C. Page & Judith Scott-Clayton, 2015. "Improving College Access in the United States: Barriers and Policy Responses," NBER Working Papers 21781, National Bureau of Economic Research, Inc.
    21. Daniel Gubits & Judy Geyer & Denise Hoffman & Sarah Croake & Utsav Kattel & David Judkins & Stephen Bell & David Stapleton, "undated". "BOND Implementation and Evaluation: 2015 Stage 2 Interim Process, Participation, and Impact Report," Mathematica Policy Research Reports 933095e3c2b74e388a05cd030, Mathematica Policy Research.
    22. Yuta Ota & Takahiro Hoshino & Taisuke Otsu, 2024. "Causal Inference With Auxiliary Observations," Keio-IES Discussion Paper Series 2024-022, Institute for Economics Studies, Keio University.
    23. Dana Rotz & Dara Lee Luca & Brian Goesling & Elizabeth Cook & Kelly Murphy & Jack Stevens, "undated". "Final Impacts of the Teen Options to Prevent Pregnancy Program," Mathematica Policy Research Reports 041f7ac88037417baa4ed3e4c, Mathematica Policy Research.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:mpr:mprres:eedf2eac4d4c4d8e869052c1d4b2d566. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joanne Pfleiderer or Cindy George The email address of this maintainer does not seem to be valid anymore. Please ask Cindy George to update the entry or send us the correct address (email available below). General contact details of provider: https://edirc.repec.org/data/mathius.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.