IDEAS home Printed from https://ideas.repec.org/a/bla/istatr/v88y2020i1p101-121.html

Sampling‐based Randomised Designs for Causal Inference under the Potential Outcomes Framework

Author

Listed:
  • Zach Branson
  • Tirthankar Dasgupta

Abstract

We establish the inferential properties of the mean‐difference estimator for the average treatment effect in randomised experiments where each unit in a population is randomised to one of two treatments and then units within treatment groups are randomly sampled. The properties of this estimator are well understood in the experimental design scenario where first units are randomly sampled and then treatment is randomly assigned but not for the aforementioned scenario where the sampling and treatment assignment stages are reversed. We find that the inferential properties of the mean‐difference estimator under this experimental design scenario are identical to those under the more common sample‐first‐randomise‐second design. This finding will bring some clarifications about sampling‐based randomised designs for causal inference, particularly for settings where there is a finite super‐population. Finally, we explore to what extent pre‐treatment measurements can be used to improve upon the mean‐difference estimator for this randomise‐first‐sample‐second design. Unfortunately, we find that pre‐treatment measurements are often unhelpful in improving the precision of average treatment effect estimators under this design, unless a large number of pre‐treatment measurements that are highly associative with the post‐treatment measurements can be obtained. We confirm these results using a simulation study based on a real experiment in nanomaterials.

Suggested Citation

  • Zach Branson & Tirthankar Dasgupta, 2020. "Sampling‐based Randomised Designs for Causal Inference under the Potential Outcomes Framework," International Statistical Review, International Statistical Institute, vol. 88(1), pages 101-121, April.
    • Handle: RePEc:bla:istatr:v:88:y:2020:i:1:p:101-121
    DOI: 10.1111/insr.12339
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/insr.12339
    Download Restriction: no
    File URL: https://libkey.io/10.1111/insr.12339?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Xinran Li & Peng Ding, 2017. "General Forms of Finite Population Central Limit Theorems with Applications to Causal Inference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(520), pages 1759-1769, October.
    2. Peng Ding & Avi Feller & Luke Miratrix, 2016. "Randomization inference for treatment effect variation," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(3), pages 655-671, June.
    3. Samii, Cyrus & Aronow, Peter M., 2012. "On equivalencies between design-based and regression-based variance estimators for randomized experiments," Statistics & Probability Letters, Elsevier, vol. 82(2), pages 365-370.
    4. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
    5. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    6. Middleton, Joel A., 2008. "Bias of the regression estimator for experiments using clustered random assignment," Statistics & Probability Letters, Elsevier, vol. 78(16), pages 2654-2659, November.
    7. Miratrix, Luke W. & Sekhon, Jasjeet S. & Theodoridis, Alexander G. & Campos, Luis F., 2018. "Worth Weighting? How to Think About and Use Weights in Survey Experiments," Political Analysis, Cambridge University Press, vol. 26(3), pages 275-291, July.
    8. Rahul Mukerjee & Tirthankar Dasgupta & Donald B. Rubin, 2018. "Using Standard Tools From Finite Population Sampling to Improve Causal Inference for Complex Experiments," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(522), pages 868-881, April.
    9. Coppock, Alexander, 2019. "Generalizing from Survey Experiments Conducted on Mechanical Turk: A Replication Approach," Political Science Research and Methods, Cambridge University Press, vol. 7(3), pages 613-628, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fangzhou Su & Peng Ding, 2021. "Model‐assisted analyses of cluster‐randomized experiments," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(5), pages 994-1015, November.
    2. Nicole E. Pashley & Luke W. Miratrix, 2021. "Insights on Variance Estimation for Blocked and Matched Pairs Designs," Journal of Educational and Behavioral Statistics, , vol. 46(3), pages 271-296, June.
    3. Lupparelli, Monia & Mattei, Alessandra, 2020. "Joint and marginal causal effects for binary non-independent outcomes," Journal of Multivariate Analysis, Elsevier, vol. 178(C).
    4. Brandon Hauser & Mauricio Olivares, 2025. "A primer on power and sample size calculations for randomisation inference with experimental data," Fiscal Studies, John Wiley & Sons, vol. 46(3), pages 349-371, September.
    5. Jiafeng Chen, 2021. "Nonparametric Treatment Effect Identification in School Choice," Papers 2112.03872, arXiv.org, revised Dec 2025.
    6. Haoge Chang, 2023. "Design-based Estimation Theory for Complex Experiments," Papers 2311.06891, arXiv.org, revised May 2025.
    7. Ganesh Karapakula, 2023. "Stable Probability Weighting: Large-Sample and Finite-Sample Estimation and Inference Methods for Heterogeneous Causal Effects of Multivalued Treatments Under Limited Overlap," Papers 2301.05703, arXiv.org, revised Jan 2023.
    8. Joel A. Middleton, 2021. "Unifying Design-based Inference: On Bounding and Estimating the Variance of any Linear Estimator in any Experimental Design," Papers 2109.09220, arXiv.org.
    9. Verena Lauber & Johanna Storck, 2016. "Helping with the Kids? How Family-Friendly Workplaces Affect Parental Well-Being and Behavior," Discussion Papers of DIW Berlin 1630, DIW Berlin, German Institute for Economic Research.
    10. Caloffi, Annalisa & Freo, Marzia & Ghinoi, Stefano & Mariani, Marco & Rossi, Federica, 2022. "Assessing the effects of a deliberate policy mix: The case of technology and innovation advisory services and innovation vouchers," Research Policy, Elsevier, vol. 51(6).
    11. Drini Imami & Orjon Xhoxhi & Davit Babayan & Thomas Herzfeld & Rustam Rakhmetov, 2026. "Treatment Effect Heterogeneity of Farmers Participating in Cooperatives," Journal of Agricultural Economics, Wiley Blackwell, vol. 77(1), pages 94-106, February.
    12. Shanike J. Smart & Solomon W. Polachek, 2024. "COVID-19 vaccine and risk-taking," Journal of Risk and Uncertainty, Springer, vol. 68(1), pages 25-49, February.
    13. Vincent Starck, 2025. "Improving control over unobservables with network data," Papers 2511.00612, arXiv.org, revised Jul 2026.
    14. Jonathan Fuhr & Philipp Berens & Dominik Papies, 2024. "Estimating Causal Effects with Double Machine Learning -- A Method Evaluation," Papers 2403.14385, arXiv.org, revised Apr 2024.
    15. J. R. Lockwood & Daniel F. McCaffrey, 2019. "Impact Evaluation Using Analysis of Covariance With Error-Prone Covariates That Violate Surrogacy," Evaluation Review, , vol. 43(6), pages 335-369, December.
    16. Clément de Chaisemartin, 2022. "Trading-off Bias and Variance in Stratified Experiments and in Staggered Adoption Designs, Under a Boundedness Condition on the Magnitude of the Treatment Effect," Working Papers hal-03873919, HAL.
    17. Florian Gunsilius & Yuliang Xu, 2021. "Matching for causal effects via multimarginal unbalanced optimal transport," Papers 2112.04398, arXiv.org, revised Jul 2022.
    18. Zhengyuan Zhou & Susan Athey & Stefan Wager, 2023. "Offline Multi-Action Policy Learning: Generalization and Optimization," Operations Research, INFORMS, vol. 71(1), pages 148-183, January.
    19. Borgschulte, Mark & Vogler, Jacob, 2020. "Did the ACA Medicaid expansion save lives?," Journal of Health Economics, Elsevier, vol. 72(C).
    20. Ashesh Rambachan & Jonathan Roth, 2026. "Design-Based Uncertainty for Quasi-Experiments," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 121(553), pages 477-491, January.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:istatr:v:88:y:2020:i:1:p:101-121. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/isiiinl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.