IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2311.06891.html
   My bibliography  Save this paper

Design-based Estimation Theory for Complex Experiments

Author

Listed:
  • Haoge Chang

Abstract

This paper considers the estimation of treatment effects in randomized experiments with complex experimental designs, including cases with interference between units. We develop a design-based estimation theory for arbitrary experimental designs. Our theory facilitates the analysis of many design-estimator pairs that researchers commonly employ in practice and provide procedures to consistently estimate asymptotic variance bounds. We propose new classes of estimators with favorable asymptotic properties from a design-based point of view. In addition, we propose a scalar measure of experimental complexity which can be linked to the design-based variance of the estimators. We demonstrate the performance of our estimators using simulated datasets based on an actual network experiment studying the effect of social networks on insurance adoptions.

Suggested Citation

  • Haoge Chang, 2023. "Design-based Estimation Theory for Complex Experiments," Papers 2311.06891, arXiv.org.
  • Handle: RePEc:arx:papers:2311.06891
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2311.06891
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Federico A. Bugni & Ivan A. Canay & Azeem M. Shaikh, 2018. "Inference Under Covariate-Adaptive Randomization," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(524), pages 1784-1796, October.
    2. Clément de Chaisemartin & Jaime Ramirez-Cuellar, 2024. "At What Level Should One Cluster Standard Errors in Paired and Small-Strata Experiments?," American Economic Journal: Applied Economics, American Economic Association, vol. 16(1), pages 193-212, January.
    3. Xinran Li & Peng Ding, 2017. "General Forms of Finite Population Central Limit Theorems with Applications to Causal Inference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(520), pages 1759-1769, October.
    4. Bruce Sacerdote, 2014. "Experimental and Quasi-Experimental Analysis of Peer Effects: Two Steps Forward?," Annual Review of Economics, Annual Reviews, vol. 6(1), pages 253-272, August.
    5. Charles F. Manski, 2013. "Identification of treatment response with social interactions," Econometrics Journal, Royal Economic Society, vol. 16(1), pages 1-23, February.
    6. Joel A. Middleton, 2021. "Unifying Design-based Inference: On Bounding and Estimating the Variance of any Linear Estimator in any Experimental Design," Papers 2109.09220, arXiv.org.
    7. Jing Cai & Alain De Janvry & Elisabeth Sadoulet, 2015. "Social Networks and the Decision to Insure," American Economic Journal: Applied Economics, American Economic Association, vol. 7(2), pages 81-108, April.
    8. Mengsi Gao & Peng Ding, 2023. "Causal inference in network experiments: regression-based analysis and design-based properties," Papers 2309.07476, arXiv.org, revised Nov 2023.
    9. Hudgens, Michael G. & Halloran, M. Elizabeth, 2008. "Toward Causal Inference With Interference," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 832-842, June.
    10. Heejung Bang & James M. Robins, 2005. "Doubly Robust Estimation in Missing Data and Causal Inference Models," Biometrics, The International Biometric Society, vol. 61(4), pages 962-973, December.
    11. Yuchen Hu & Shuangning Li & Stefan Wager, 2021. "Average Direct and Indirect Causal Effects under Interference," Papers 2104.03802, arXiv.org, revised Jan 2022.
    12. Lihua Lei & Peng Ding, 2021. "Regression adjustment in completely randomized experiments with a diverging number of covariates [Covariance adjustments for the analysis of randomized field experiments]," Biometrika, Biometrika Trust, vol. 108(4), pages 815-828.
    13. Jing Cai & Adam Szeidl, 2018. "Interfirm Relationships and Business Performance," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(3), pages 1229-1282.
    14. Ruonan Xu, 2021. "Potential outcomes and finite-population inference for M-estimators," The Econometrics Journal, Royal Economic Society, vol. 24(1), pages 162-176.
    15. Yuchen Hu & Shuangning Li & Stefan Wager, 2022. "Average direct and indirect causal effects under interference [Estimating average causal effects under general interference, with application to a social network experiment]," Biometrika, Biometrika Trust, vol. 109(4), pages 1165-1172.
    16. Federico A. Bugni & Ivan A. Canay & Azeem M. Shaikh, 2019. "Inference under covariate‐adaptive randomization with multiple treatments," Quantitative Economics, Econometric Society, vol. 10(4), pages 1747-1785, November.
    17. Edward Miguel & Michael Kremer, 2004. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities," Econometrica, Econometric Society, vol. 72(1), pages 159-217, January.
    18. Akanksha Negi & Jeffrey M. Wooldridge, 2021. "Revisiting regression adjustment in experiments with heterogeneous treatment effects," Econometric Reviews, Taylor & Francis Journals, vol. 40(5), pages 504-534, April.
    19. Jason Wu & Peng Ding, 2021. "Randomization Tests for Weak Null Hypotheses in Randomized Experiments," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1898-1913, October.
    20. Luke W. Miratrix & Jasjeet S. Sekhon & Bin Yu, 2013. "Adjusting treatment effect estimates by post-stratification in randomized experiments," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(2), pages 369-396, March.
    21. Dennis Egger & Johannes Haushofer & Edward Miguel & Paul Niehaus & Michael Walker, 2022. "General Equilibrium Effects of Cash Transfers: Experimental Evidence From Kenya," Econometrica, Econometric Society, vol. 90(6), pages 2603-2643, November.
    22. Lorenzo Fattorini, 2006. "Applying the Horvitz-Thompson criterion in complex designs: A computer-intensive perspective for estimating inclusion probabilities," Biometrika, Biometrika Trust, vol. 93(2), pages 269-278, June.
    23. Athey, Susan & Imbens, Guido W., 2022. "Design-based analysis in Difference-In-Differences settings with staggered adoption," Journal of Econometrics, Elsevier, vol. 226(1), pages 62-79.
    24. Michael P. Leung, 2022. "Causal Inference Under Approximate Neighborhood Interference," Econometrica, Econometric Society, vol. 90(1), pages 267-293, January.
    25. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2020. "Sampling‐Based versus Design‐Based Uncertainty in Regression Analysis," Econometrica, Econometric Society, vol. 88(1), pages 265-296, January.
    26. Rahul Mukerjee & Tirthankar Dasgupta & Donald B. Rubin, 2018. "Using Standard Tools From Finite Population Sampling to Improve Causal Inference for Complex Experiments," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(522), pages 868-881, April.
    27. Ruonan Xu & Jeffrey M. Wooldridge, 2022. "A Design-Based Approach to Spatial Correlation," Papers 2211.14354, arXiv.org.
    28. Rothenberg, Thomas J, 1971. "Identification in Parametric Models," Econometrica, Econometric Society, vol. 39(3), pages 577-591, May.
    29. Colin B Fogarty, 2018. "Regression-assisted inference for the average treatment effect in paired experiments," Biometrika, Biometrika Trust, vol. 105(4), pages 994-1000.
    30. Nicole E. Pashley & Luke W. Miratrix, 2021. "Insights on Variance Estimation for Blocked and Matched Pairs Designs," Journal of Educational and Behavioral Statistics, , vol. 46(3), pages 271-296, June.
    31. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yuehao Bai & Azeem M. Shaikh & Max Tabord-Meehan, 2024. "A Primer on the Analysis of Randomized Experiments and a Survey of some Recent Advances," Papers 2405.03910, arXiv.org.
    2. Zhao, Anqi & Ding, Peng, 2024. "No star is good news: A unified look at rerandomization based on p-values from covariate balance tests," Journal of Econometrics, Elsevier, vol. 241(1).
    3. Tadao Hoshino & Takahide Yanagi, 2021. "Causal Inference with Noncompliance and Unknown Interference," Papers 2108.07455, arXiv.org, revised Oct 2023.
    4. Han, Kevin & Basse, Guillaume & Bojinov, Iavor, 2024. "Population interference in panel experiments," Journal of Econometrics, Elsevier, vol. 238(1).
    5. Laura Forastiere & Davide Del Prete & Valerio Leone Sciabolazza, 2020. "Causal Inference on Networks under Continuous Treatment Interference," Papers 2004.13459, arXiv.org, revised Jun 2023.
    6. Zhao, Anqi & Ding, Peng, 2021. "Covariate-adjusted Fisher randomization tests for the average treatment effect," Journal of Econometrics, Elsevier, vol. 225(2), pages 278-294.
    7. Christopher Harshaw & Fredrik Savje & Yitan Wang, 2022. "A Design-Based Riesz Representation Framework for Randomized Experiments," Papers 2210.08698, arXiv.org, revised Oct 2022.
    8. Jinglong Zhao, 2024. "Experimental Design For Causal Inference Through An Optimization Lens," Papers 2408.09607, arXiv.org, revised Aug 2024.
    9. Jiang, Liang & Phillips, Peter C.B. & Tao, Yubo & Zhang, Yichong, 2023. "Regression-adjusted estimation of quantile treatment effects under covariate-adaptive randomizations," Journal of Econometrics, Elsevier, vol. 234(2), pages 758-776.
    10. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Jun 2024.
    11. Ganesh Karapakula, 2023. "Stable Probability Weighting: Large-Sample and Finite-Sample Estimation and Inference Methods for Heterogeneous Causal Effects of Multivalued Treatments Under Limited Overlap," Papers 2301.05703, arXiv.org, revised Jan 2023.
    12. Liang Jiang & Oliver B. Linton & Haihan Tang & Yichong Zhang, 2022. "Improving Estimation Efficiency via Regression-Adjustment in Covariate-Adaptive Randomizations with Imperfect Compliance," Papers 2201.13004, arXiv.org, revised Jun 2023.
    13. Davide Viviano & Jess Rudder, 2020. "Policy design in experiments with unknown interference," Papers 2011.08174, arXiv.org, revised May 2024.
    14. Ruonan Xu, 2023. "Difference-in-Differences with Interference," Papers 2306.12003, arXiv.org, revised May 2024.
    15. Davide Viviano, 2020. "Experimental Design under Network Interference," Papers 2003.08421, arXiv.org, revised Jul 2022.
    16. Fangzhou Su & Peng Ding, 2021. "Model‐assisted analyses of cluster‐randomized experiments," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(5), pages 994-1015, November.
    17. David M. Ritzwoller & Joseph P. Romano & Azeem M. Shaikh, 2024. "Randomization Inference: Theory and Applications," Papers 2406.09521, arXiv.org.
    18. Yi Zhang & Kosuke Imai, 2023. "Individualized Policy Evaluation and Learning under Clustered Network Interference," Papers 2311.02467, arXiv.org, revised Feb 2024.
    19. Luofeng Liao & Christian Kroer, 2024. "Statistical Inference and A/B Testing in Fisher Markets and Paced Auctions," Papers 2406.15522, arXiv.org, revised Aug 2024.
    20. Chabé-Ferret, Sylvain & Reynaud, Arnaud & Tène, Eva, 2021. "Water Quality, Policy Diffusion Effects and Farmers’ Behavior," TSE Working Papers 21-1229, Toulouse School of Economics (TSE).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2311.06891. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.