IDEAS home Printed from https://ideas.repec.org/a/bpj/causin/v4y2016i1p61-80n3.html

A Conditional Randomization Test to Account for Covariate Imbalance in Randomized Experiments

Author

Listed:
  • Hennessy Jonathan

    (Department of Statistics, Harvard University, Cambridge, MA, USA)

  • Dasgupta Tirthankar

    (Department of Statistics, Harvard University, Cambridge, MA, USA)

  • Miratrix Luke

    (Department of Statistics, Harvard University, Cambridge, MA, USA)

  • Pattanayak Cassandra

    (Quantitative Analysis Institute, Wellesley College, Wellesley, MA, USA)

  • Sarkar Pradipta

    (Principal Scientist, Procter & Gamble International Operations, Singapore)

Abstract

We consider the conditional randomization test as a way to account for covariate imbalance in randomized experiments. The test accounts for covariate imbalance by comparing the observed test statistic to the null distribution of the test statistic conditional on the observed covariate imbalance. We prove that the conditional randomization test has the correct significance level and introduce original notation to describe covariate balance more formally. Through simulation, we verify that conditional randomization tests behave like more traditional forms of covariate adjustment but have the added benefit of having the correct conditional significance level. Finally, we apply the approach to a randomized product marketing experiment where covariate information was collected after randomization.

Suggested Citation

  • Hennessy Jonathan & Dasgupta Tirthankar & Miratrix Luke & Pattanayak Cassandra & Sarkar Pradipta, 2016. "A Conditional Randomization Test to Account for Covariate Imbalance in Randomized Experiments," Journal of Causal Inference, De Gruyter, vol. 4(1), pages 61-80, March.
    • Handle: RePEc:bpj:causin:v:4:y:2016:i:1:p:61-80:n:3
    DOI: 10.1515/jci-2015-0018
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/jci-2015-0018
    Download Restriction: no
    File URL: https://libkey.io/10.1515/jci-2015-0018?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Min Zhang & Anastasios A. Tsiatis & Marie Davidian, 2008. "Improving Efficiency of Inferences in Randomized Clinical Trials Using Auxiliary Covariates," Biometrics, The International Biometric Society, vol. 64(3), pages 707-715, September.
    2. Luke W. Miratrix & Jasjeet S. Sekhon & Bin Yu, 2013. "Adjusting treatment effect estimates by post-stratification in randomized experiments," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(2), pages 369-396, March.
    3. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Guillaume Basse & Peng Ding & Avi Feller & Panos Toulis, 2024. "Randomization Tests for Peer Effects in Group Formation Experiments," Econometrica, Econometric Society, vol. 92(2), pages 567-590, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Donald P. Green & Winston Lin & Claudia Gerber, 2018. "Optimal Allocation of Interviews to Baseline and Endline Surveys in Place-Based Randomized Trials and Quasi-Experiments," Evaluation Review, , vol. 42(4), pages 391-422, August.
    2. Middleton Joel A. & Aronow Peter M., 2015. "Unbiased Estimation of the Average Treatment Effect in Cluster-Randomized Experiments," Statistics, Politics and Policy, De Gruyter, vol. 6(1-2), pages 39-75, December.
    3. Peter Z. Schochet, "undated". "Statistical Theory for the RCT-YES Software: Design-Based Causal Inference for RCTs," Mathematica Policy Research Reports a0c005c003c242308a92c02dc, Mathematica Policy Research.
    4. Zhao, Anqi & Ding, Peng, 2021. "Covariate-adjusted Fisher randomization tests for the average treatment effect," Journal of Econometrics, Elsevier, vol. 225(2), pages 278-294.
    5. Gabriele Ruiu & Giovanna Gonano, 2020. "Religious Barriers to the Diffusion of Same-sex Civil Unions in Italy," Population Research and Policy Review, Springer;Southern Demographic Association (SDA), vol. 39(6), pages 1185-1203, December.
    6. Wright, Austin L. & Sonin, Konstantin & Driscoll, Jesse & Wilson, Jarnickae, 2020. "Poverty and economic dislocation reduce compliance with COVID-19 shelter-in-place protocols," Journal of Economic Behavior & Organization, Elsevier, vol. 180(C), pages 544-554.
    7. Guido de Blasio & Daniela Vuri, 2019. "Effects of the Joint Custody Law in Italy," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 16(3), pages 479-514, September.
    8. Graves Jennifer & McMullen Steven & Rouse Kathryn, 2018. "Teacher Turnover, Composition and Qualifications in the Year-Round School Setting," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 18(3), pages 1-27, July.
    9. Alston Lee J. & Mueller Bernardo, 2018. "Priests, Conflicts and Property Rights: the Impacts on Tenancy and Land Use in Brazil," Man and the Economy, De Gruyter, vol. 5(1), pages 1-26, June.
    10. S Anukriti & Catalina Herrera‐Almanza & Praveen K. Pathak & Mahesh Karra, 2020. "Curse of the Mummy‐ji: The Influence of Mothers‐in‐Law on Women in India†," American Journal of Agricultural Economics, John Wiley & Sons, vol. 102(5), pages 1328-1351, October.
    11. Ellison, Richard B. & Ellison, Adrian B. & Greaves, Stephen P. & Sampaio, Breno, 2017. "Electronic ticketing systems as a mechanism for travel behaviour change? Evidence from Sydney’s Opal card," Transportation Research Part A: Policy and Practice, Elsevier, vol. 99(C), pages 80-93.
    12. Cruzatti C., John & Bjørnskov, Christian & Sáenz de Viteri, Andrea & Cruzatti, Christian, 2024. "Geography, development, and power: Parliament leaders and local clientelism," World Development, Elsevier, vol. 182(C).
    13. Yusuke Matsuki, 2016. "A Distribution-Free Test of Monotonicity with an Application to Auctions," Working Papers e110, Tokyo Center for Economic Research.
    14. Shvartsman, Elena & Beckmann, Michael, 2015. "Stressed by your job: What is the role of personnel policy?," Working papers 2015/15, Faculty of Business and Economics - University of Basel.
    15. MacDonald, Peter, 2013. "Labour substitution and the scope for military outsourcing," MPRA Paper 46688, University Library of Munich, Germany.
    16. Matteo Migheli, 2021. "Green purchasing: the effect of parenthood and gender," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 23(7), pages 10576-10600, July.
    17. Yuyang Li & Jiahui Li & Xinjie Li & Qian Lu, 2024. "Does Participation in Digital Supply and Marketing Promote Smallholder Farmers’ Adoption of Green Agricultural Production Technologies?," Land, MDPI, vol. 14(1), pages 1-24, December.
    18. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    19. Sondang Marsinta Uli Panggabean & Mahjus Ekananda & Beta Yulianita Gitaharie & Leslie Djuranovik, 2025. "Export proceeds repatriation policies: A shield against exchange rate volatility in emerging markets?," Papers 2506.09168, arXiv.org.
    20. Laurent Didier, 2017. "South-South Trade and Geographical Diversification of Intra-SSA Trade: Evidence from BRICs," African Development Review, African Development Bank, vol. 29(2), pages 139-154, June.

    More about this item

    Keywords

    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:causin:v:4:y:2016:i:1:p:61-80:n:3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyterbrill.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.