IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2301.09016.html
   My bibliography  Save this paper

Inference for Two-stage Experiments under Covariate-Adaptive Randomization

Author

Listed:
  • Jizhou Liu

Abstract

This paper studies inference in two-stage randomized experiments under covariate-adaptive randomization. In the initial stage of this experimental design, clusters (e.g., households, schools, or graph partitions) are stratified and randomly assigned to control or treatment groups based on cluster-level covariates. Subsequently, an independent second-stage design is carried out, wherein units within each treated cluster are further stratified and randomly assigned to either control or treatment groups, based on individual-level covariates. Under the homogeneous partial interference assumption, I establish conditions under which the proposed difference-in-"average of averages" estimators are consistent and asymptotically normal for the corresponding average primary and spillover effects and develop consistent estimators of their asymptotic variances. Combining these results establishes the asymptotic validity of tests based on these estimators. My findings suggest that ignoring covariate information in the design stage can result in efficiency loss, and commonly used inference methods that ignore or improperly use covariate information can lead to either conservative or invalid inference. Finally, I apply these results to studying optimal use of covariate information under covariate-adaptive randomization in large samples, and demonstrate that a specific generalized matched-pair design achieves minimum asymptotic variance for each proposed estimator. The practical relevance of the theoretical results is illustrated through a simulation study and an empirical application.

Suggested Citation

  • Jizhou Liu, 2023. "Inference for Two-stage Experiments under Covariate-Adaptive Randomization," Papers 2301.09016, arXiv.org, revised Oct 2023.
  • Handle: RePEc:arx:papers:2301.09016
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2301.09016
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Federico A. Bugni & Ivan A. Canay & Azeem M. Shaikh, 2018. "Inference Under Covariate-Adaptive Randomization," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(524), pages 1784-1796, October.
    2. Federico Bugni & Ivan Canay & Azeem Shaikh & Max Tabord-Meehan, 2022. "Inference for Cluster Randomized Experiments with Non-ignorable Cluster Sizes," Papers 2204.08356, arXiv.org, revised Apr 2024.
    3. Abhijit Banerjee & Raghabendra Chattopadhyay & Esther Duflo & Daniel Keniston & Nina Singh, 2021. "Improving Police Performance in Rajasthan, India: Experimental Evidence on Incentives, Managerial Autonomy, and Training," American Economic Journal: Economic Policy, American Economic Association, vol. 13(1), pages 36-66, February.
    4. Suresh de Mel & David McKenzie & Christopher Woodruff, 2013. "The Demand for, and Consequences of, Formalization among Informal Firms in Sri Lanka," American Economic Journal: Applied Economics, American Economic Association, vol. 5(2), pages 122-150, April.
    5. Florian Foos & Eline A. de Rooij, 2017. "All in the Family: Partisan Disagreement and Electoral Mobilization in Intimate Networks—A Spillover Experiment," American Journal of Political Science, John Wiley & Sons, vol. 61(2), pages 289-304, April.
    6. Esther Duflo & Emmanuel Saez, 2003. "The Role of Information and Social Interactions in Retirement Plan Decisions: Evidence from a Randomized Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(3), pages 815-842.
    7. G W Basse & A Feller & P Toulis, 2019. "Randomization tests of causal effects under interference," Biometrika, Biometrika Trust, vol. 106(2), pages 487-494.
    8. Kosuke Imai & Zhichao Jiang & Anup Malani, 2021. "Causal Inference With Interference and Noncompliance in Two-Stage Randomized Experiments," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(534), pages 632-644, April.
    9. Hudgens, Michael G. & Halloran, M. Elizabeth, 2008. "Toward Causal Inference With Interference," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 832-842, June.
    10. David McKenzie & Susana Puerto, 2021. "Growing Markets through Business Training for Female Entrepreneurs: A Market-Level Randomized Experiment in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 13(2), pages 297-332, April.
    11. Fafchamps, Marcel & McKenzie, David & Quinn, Simon & Woodruff, Christopher, 2014. "Microenterprise growth and the flypaper effect: Evidence from a randomized experiment in Ghana," Journal of Development Economics, Elsevier, vol. 106(C), pages 211-226.
    12. Bold, Tessa & Kimenyi, Mwangi & Mwabu, Germano & Ng’ang’a, Alice & Sandefur, Justin, 2018. "Experimental evidence on scaling up education reforms in Kenya," Journal of Public Economics, Elsevier, vol. 168(C), pages 1-20.
    13. Gonzalo Vazquez-Bare, 2017. "Identification and Estimation of Spillover Effects in Randomized Experiments," Papers 1711.02745, arXiv.org, revised Jan 2022.
    14. Johannes Haushofer & Jeremy Shapiro, 2016. "The Short-term Impact of Unconditional Cash Transfers to the Poor: ExperimentalEvidence from Kenya," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(4), pages 1973-2042.
    15. Aramburu, Julián & Figal Garone, Lucas & Maffioli, Alessandro & Salazar, Lina & López, César Augusto, 2019. "Direct and Spillover Effects of Agricultural Technology Adoption Programs: Experimental Evidence from the Dominican Republic," IDB Publications (Working Papers) 9671, Inter-American Development Bank.
    16. Yuehao Bai, 2022. "Optimality of Matched-Pair Designs in Randomized Controlled Trials," American Economic Review, American Economic Association, vol. 112(12), pages 3911-3940, December.
    17. Guillermo Cruces & Dario Tortarolo & Gonzalo Vazquez-Bare, 2022. "Design of two-stage experiments with an application to spillovers in tax compliance," IFS Working Papers W22/32, Institute for Fiscal Studies.
    18. Laura Forastiere & Edoardo M. Airoldi & Fabrizia Mealli, 2021. "Identification and Estimation of Treatment and Interference Effects in Observational Studies on Networks," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(534), pages 901-918, April.
    19. Rigdon, Joseph & Hudgens, Michael G., 2015. "Exact confidence intervals in the presence of interference," Statistics & Probability Letters, Elsevier, vol. 105(C), pages 130-135.
    20. Lan Liu & Michael G. Hudgens, 2014. "Large Sample Randomization Inference of Causal Effects in the Presence of Interference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(505), pages 288-301, March.
    21. Melissa Hidrobo & Amber Peterman & Lori Heise, 2016. "The Effect of Cash, Vouchers, and Food Transfers on Intimate Partner Violence: Evidence from a Randomized Experiment in Northern Ecuador," American Economic Journal: Applied Economics, American Economic Association, vol. 8(3), pages 284-303, July.
    22. Diether W. Beuermann & Julian Cristia & Santiago Cueto & Ofer Malamud & Yyannu Cruz-Aguayo, 2015. "One Laptop per Child at Home: Short-Term Impacts from a Randomized Experiment in Peru," American Economic Journal: Applied Economics, American Economic Association, vol. 7(2), pages 53-80, April.
    23. Johannes Haushofer & Charlotte Ringdal & Jeremy P. Shapiro & Xiao Yu Wang, 2019. "Income Changes and Intimate Partner Violence: Evidence from Unconditional Cash Transfers in Kenya," NBER Working Papers 25627, National Bureau of Economic Research, Inc.
    24. Todd Rogers & Avi Feller, 2018. "Reducing student absences at scale by targeting parents’ misbeliefs," Nature Human Behaviour, Nature, vol. 2(5), pages 335-342, May.
    25. Yuehao Bai, 2022. "Optimality of Matched-Pair Designs in Randomized Controlled Trials," Papers 2206.07845, arXiv.org.
    26. Guillaume Basse & Avi Feller, 2018. "Analyzing Two-Stage Experiments in the Presence of Interference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(521), pages 41-55, January.
    27. Karthik Muralidharan & Venkatesh Sundararaman, 2015. "Editor's Choice The Aggregate Effect of School Choice: Evidence from a Two-Stage Experiment in India," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(3), pages 1011-1066.
    28. Yuehao Bai & Jizhou Liu & Max Tabord-Meehan, 2022. "Inference for Matched Tuples and Fully Blocked Factorial Designs," Papers 2206.04157, arXiv.org, revised Nov 2023.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guillermo Cruces & Dario Tortarolo & Gonzalo Vazquez-Bare, 2022. "Design of two-stage experiments with an application to spillovers in tax compliance," IFS Working Papers W22/32, Institute for Fiscal Studies.
    2. Yuehao Bai & Jizhou Liu & Max Tabord-Meehan, 2022. "Inference for Matched Tuples and Fully Blocked Factorial Designs," Papers 2206.04157, arXiv.org, revised Nov 2023.
    3. Gonzalo Vazquez-Bare, 2017. "Identification and Estimation of Spillover Effects in Randomized Experiments," Papers 1711.02745, arXiv.org, revised Jan 2022.
    4. Gonzalo Vazquez-Bare, 2020. "Causal Spillover Effects Using Instrumental Variables," Papers 2003.06023, arXiv.org, revised Dec 2021.
    5. Dario Tortarolo & Guillermo Cruces & Gonzalo Vazquez-Bare, 2023. "Design of partial population experiments with an application to spillovers in tax compliance," IFS Working Papers W23/17, Institute for Fiscal Studies.
    6. Zhaonan Qu & Ruoxuan Xiong & Jizhou Liu & Guido Imbens, 2021. "Efficient Treatment Effect Estimation in Observational Studies under Heterogeneous Partial Interference," Papers 2107.12420, arXiv.org, revised Jun 2022.
    7. Yuehao Bai & Meng Hsuan Hsieh & Jizhou Liu & Max Tabord-Meehan, 2022. "Revisiting the Analysis of Matched-Pair and Stratified Experiments in the Presence of Attrition," Papers 2209.11840, arXiv.org, revised Oct 2023.
    8. Tadao Hoshino & Takahide Yanagi, 2021. "Causal Inference with Noncompliance and Unknown Interference," Papers 2108.07455, arXiv.org, revised Oct 2023.
    9. Zhichao Jiang & Kosuke Imai & Anup Malani, 2023. "Statistical inference and power analysis for direct and spillover effects in two‐stage randomized experiments," Biometrics, The International Biometric Society, vol. 79(3), pages 2370-2381, September.
    10. Peter Bergman, 2020. "Nudging Technology Use: Descriptive and Experimental Evidence from School Information Systems," Education Finance and Policy, MIT Press, vol. 15(4), pages 623-647, Fall.
    11. Yi Zhang & Kosuke Imai, 2023. "Individualized Policy Evaluation and Learning under Clustered Network Interference," Papers 2311.02467, arXiv.org, revised Feb 2024.
    12. Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," SciencePo Working papers Main hal-03455978, HAL.
    13. Jiang, Liang & Phillips, Peter C.B. & Tao, Yubo & Zhang, Yichong, 2023. "Regression-adjusted estimation of quantile treatment effects under covariate-adaptive randomizations," Journal of Econometrics, Elsevier, vol. 234(2), pages 758-776.
    14. DiTraglia, Francis J. & García-Jimeno, Camilo & O’Keeffe-O’Donovan, Rossa & Sánchez-Becerra, Alejandro, 2023. "Identifying causal effects in experiments with spillovers and non-compliance," Journal of Econometrics, Elsevier, vol. 235(2), pages 1589-1624.
    15. Francis J. DiTraglia & Camilo Garcia-Jimeno & Rossa O'Keeffe-O'Donovan & Alejandro Sanchez-Becerra, 2020. "Identifying Causal Effects in Experiments with Spillovers and Non-compliance," Papers 2011.07051, arXiv.org, revised Jan 2023.
    16. Sarah Baird & Aislinn Bohren & Craig McIntosh & Berk Ozler, 2014. "Designing Experiments to Measure Spillover Effects," PIER Working Paper Archive 14-032, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
    17. Liang Jiang & Xiaobin Liu & Peter C. B. Phillips & Yichong Zhang, 2020. "Bootstrap Inference for Quantile Treatment Effects in Randomized Experiments with Matched Pairs," Papers 2005.11967, arXiv.org, revised May 2021.
    18. Shaina J. Alexandria & Michael G. Hudgens & Allison E. Aiello, 2023. "Assessing intervention effects in a randomized trial within a social network," Biometrics, The International Biometric Society, vol. 79(2), pages 1409-1419, June.
    19. David McKenzie & Anna Luisa Paffhausen, 2019. "Small Firm Death in Developing Countries," The Review of Economics and Statistics, MIT Press, vol. 101(4), pages 645-657, October.
    20. Campos, Francisco & Goldstein, Markus & McKenzie, David, 2023. "How should the government bring small firms into the formal system? Experimental evidence from Malawi," Journal of Development Economics, Elsevier, vol. 161(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2301.09016. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.