IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v42y2018i4p458-488.html
   My bibliography  Save this article

The LOOP Estimator: Adjusting for Covariates in Randomized Experiments

Author

Listed:
  • Edward Wu
  • Johann A. Gagnon-Bartsch

Abstract

Background: When conducting a randomized controlled trial, it is common to specify in advance the statistical analyses that will be used to analyze the data. Typically, these analyses will involve adjusting for small imbalances in baseline covariates. However, this poses a dilemma, as adjusting for too many covariates can hurt precision more than it helps, and it is often unclear which covariates are predictive of outcome prior to conducting the experiment. Objectives: This article aims to produce a covariate adjustment method that allows for automatic variable selection, so that practitioners need not commit to any specific set of covariates prior to seeing the data. Results: In this article, we propose the “leave-one-out potential outcomes†estimator. We leave out each observation and then impute that observation’s treatment and control potential outcomes using a prediction algorithm such as a random forest. In addition to allowing for automatic variable selection, this estimator is unbiased under the Neyman–Rubin model, generally performs at least as well as the unadjusted estimator, and the experimental randomization largely justifies the statistical assumptions made.

Suggested Citation

  • Edward Wu & Johann A. Gagnon-Bartsch, 2018. "The LOOP Estimator: Adjusting for Covariates in Randomized Experiments," Evaluation Review, , vol. 42(4), pages 458-488, August.
    • Handle: RePEc:sae:evarev:v:42:y:2018:i:4:p:458-488
    DOI: 10.1177/0193841X18808003
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X18808003
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X18808003?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey & James Robins, 2018. "Double/debiased machine learning for treatment and structural parameters," Econometrics Journal, Royal Economic Society, vol. 21(1), pages 1-68, February.
    2. Luke W. Miratrix & Jasjeet S. Sekhon & Bin Yu, 2013. "Adjusting treatment effect estimates by post-stratification in randomized experiments," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(2), pages 369-396, March.
    3. Kenneth F Schulz & Douglas G Altman & David Moher & for the CONSORT Group, 2010. "CONSORT 2010 Statement: Updated Guidelines for Reporting Parallel Group Randomised Trials," PLOS Medicine, Public Library of Science, vol. 7(3), pages 1-7, March.
    4. Felipe Barrera-Osorio & Marianne Bertrand & Leigh L. Linden & Francisco Perez-Calle, 2011. "Improving the Design of Conditional Transfer Programs: Evidence from a Randomized Education Experiment in Colombia," American Economic Journal: Applied Economics, American Economic Association, vol. 3(2), pages 167-195, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jörg Peters & Jörg Langbein & Gareth Roberts, 2018. "Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity," The World Bank Research Observer, World Bank, vol. 33(1), pages 34-64.
    2. Cai, Yunhao & Jing, Peng & Wang, Baihui & Jiang, Chengxi & Wang, Yuan, 2023. "How does “over-hype” lead to public misconceptions about autonomous vehicles? A new insight applying causal inference," Transportation Research Part A: Policy and Practice, Elsevier, vol. 175(C).
    3. Tiziano Arduini & Eleonora Patacchini & Edoardo Rainone, 2020. "Treatment Effects With Heterogeneous Externalities," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(4), pages 826-838, October.
    4. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    5. Robert Darko Osei & Kwabena Adu-Ababio, 2018. "Effects of an education reform on household poverty and inequality: A microsimulation analysis on the free Senior High School policy in Ghana," WIDER Working Paper Series 147, World Institute for Development Economic Research (UNU-WIDER).
    6. Ángel Enrique & Juana Bretón-López & Guadalupe Molinari & Rosa M. Baños & Cristina Botella, 2018. "Efficacy of an adaptation of the Best Possible Self intervention implemented through positive technology: a randomized control trial," Applied Research in Quality of Life, Springer;International Society for Quality-of-Life Studies, vol. 13(3), pages 671-689, September.
    7. Gerben ter Riet & Paula Chesley & Alan G Gross & Lara Siebeling & Patrick Muggensturm & Nadine Heller & Martin Umbehr & Daniela Vollenweider & Tsung Yu & Elie A Akl & Lizzy Brewster & Olaf M Dekkers &, 2013. "All That Glitters Isn't Gold: A Survey on Acknowledgment of Limitations in Biomedical Studies," PLOS ONE, Public Library of Science, vol. 8(11), pages 1-6, November.
    8. Davide Viviano & Jelena Bradic, 2019. "Synthetic learner: model-free inference on treatments over time," Papers 1904.01490, arXiv.org, revised Aug 2022.
    9. Spyridon N Papageorgiou & Georgios N Antonoglou & George K Sándor & Theodore Eliades, 2017. "Randomized clinical trials in orthodontics are rarely registered a priori and often published late or not at all," PLOS ONE, Public Library of Science, vol. 12(8), pages 1-13, August.
    10. Eun-Hi Kong & Myoungsuk Kim & Seonho Kim, 2021. "Effects of a Web-Based Educational Program Regarding Physical Restraint Reduction in Long-Term Care Settings on Nursing Students: A Cluster Randomized Controlled Trial," IJERPH, MDPI, vol. 18(13), pages 1-10, June.
    11. Sung Jae Jun & Sokbae Lee, 2020. "Causal Inference under Outcome-Based Sampling with Monotonicity Assumptions," Papers 2004.08318, arXiv.org, revised Oct 2023.
    12. Stavros Petrou & Oliver Rivero-Arias & Helen Dakin & Louise Longworth & Mark Oppe & Robert Froud & Alastair Gray, 2015. "Preferred Reporting Items for Studies Mapping onto Preference-Based Outcome Measures: The MAPS Statement," Medical Decision Making, , vol. 35(6), pages 1-8, August.
    13. Alexander P. L. Martindale & Benjamin Ng & Victoria Ngai & Aditya U. Kale & Lavinia Ferrante di Ruffano & Robert M. Golub & Gary S. Collins & David Moher & Melissa D. McCradden & Lauren Oakden-Rayner , 2024. "Concordance of randomised controlled trials for artificial intelligence interventions with the CONSORT-AI reporting guidelines," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    14. Di Giannatale, Sonia & Roa, María José, 2016. "Formal Saving in Developing Economies: Barriers, Interventions, and Effects," IDB Publications (Working Papers) 8107, Inter-American Development Bank.
    15. Munday, Tim & Brookes, James, 2021. "Mark my words: the transmission of central bank communication to the general public via the print media," Bank of England working papers 944, Bank of England.
    16. Juan Carlos Escanciano & Telmo P'erez-Izquierdo, 2023. "Automatic Locally Robust Estimation with Generated Regressors," Papers 2301.10643, arXiv.org, revised Nov 2023.
    17. M. Caridad Araujo & Mariano Bosch & Norbert Schady, 2017. "Can Cash Transfers Help Households Escape an Intergenerational Poverty Trap?," NBER Chapters, in: The Economics of Poverty Traps, pages 357-382, National Bureau of Economic Research, Inc.
    18. Elek, Péter & Bíró, Anikó, 2021. "Regional differences in diabetes across Europe – regression and causal forest analyses," Economics & Human Biology, Elsevier, vol. 40(C).
    19. Michael C. Knaus, 2021. "A double machine learning approach to estimate the effects of musical practice on student’s skills," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(1), pages 282-300, January.
    20. Felipe Barrera-Osorio & Leigh L. Linden & Juan E. Saavedra, 2019. "Medium- and Long-Term Educational Consequences of Alternative Conditional Cash Transfer Designs: Experimental Evidence from Colombia," American Economic Journal: Applied Economics, American Economic Association, vol. 11(3), pages 54-91, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:42:y:2018:i:4:p:458-488. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.