IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2112.13495.html
   My bibliography  Save this paper

Multiple Randomization Designs

Author

Listed:
  • Patrick Bajari
  • Brian Burdick
  • Guido W. Imbens
  • Lorenzo Masoero
  • James McQueen
  • Thomas Richardson
  • Ido M. Rosen

Abstract

In this study we introduce a new class of experimental designs. In a classical randomized controlled trial (RCT), or A/B test, a randomly selected subset of a population of units (e.g., individuals, plots of land, or experiences) is assigned to a treatment (treatment A), and the remainder of the population is assigned to the control treatment (treatment B). The difference in average outcome by treatment group is an estimate of the average effect of the treatment. However, motivating our study, the setting for modern experiments is often different, with the outcomes and treatment assignments indexed by multiple populations. For example, outcomes may be indexed by buyers and sellers, by content creators and subscribers, by drivers and riders, or by travelers and airlines and travel agents, with treatments potentially varying across these indices. Spillovers or interference can arise from interactions between units across populations. For example, sellers' behavior may depend on buyers' treatment assignment, or vice versa. This can invalidate the simple comparison of means as an estimator for the average effect of the treatment in classical RCTs. We propose new experiment designs for settings in which multiple populations interact. We show how these designs allow us to study questions about interference that cannot be answered by classical randomized experiments. Finally, we develop new statistical methods for analyzing these Multiple Randomization Designs.

Suggested Citation

  • Patrick Bajari & Brian Burdick & Guido W. Imbens & Lorenzo Masoero & James McQueen & Thomas Richardson & Ido M. Rosen, 2021. "Multiple Randomization Designs," Papers 2112.13495, arXiv.org.
    • Handle: RePEc:arx:papers:2112.13495
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2112.13495
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ruoxuan Xiong & Susan Athey & Mohsen Bayati & Guido Imbens, 2019. "Optimal Experimental Design for Staggered Rollouts," Papers 1911.03764, arXiv.org, revised Sep 2023.
    2. Evan Munro & Stefan Wager & Kuang Xu, 2021. "Treatment Effects in Market Equilibrium," Papers 2109.11647, arXiv.org, revised Jan 2023.
    3. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Eric Auerbach & Yong Cai, 2022. "Heterogeneous Treatment Effects for Networks, Panels, and other Outcome Matrices," Papers 2205.01246, arXiv.org, revised Oct 2022.
    2. Evan Munro & David Jones & Jennifer Brennan & Roland Nelet & Vahab Mirrokni & Jean Pouget-Abadie, 2023. "Causal Estimation of User Learning in Personalized Systems," Papers 2306.00485, arXiv.org.
    3. Ramesh Johari & Hannah Li & Inessa Liskovich & Gabriel Y. Weintraub, 2022. "Experimental Design in Two-Sided Platforms: An Analysis of Bias," Management Science, INFORMS, vol. 68(10), pages 7069-7089, October.
    4. Anish Agarwal & Sarah H. Cen & Devavrat Shah & Christina Lee Yu, 2022. "Network Synthetic Interventions: A Causal Framework for Panel Data Under Network Interference," Papers 2210.11355, arXiv.org, revised Oct 2023.
    5. Guido W. Imbens, 2022. "Causality in Econometrics: Choice vs Chance," Econometrica, Econometric Society, vol. 90(6), pages 2541-2566, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Athey, Susan & Imbens, Guido W., 2022. "Design-based analysis in Difference-In-Differences settings with staggered adoption," Journal of Econometrics, Elsevier, vol. 226(1), pages 62-79.
    2. Iavor Bojinov & David Simchi-Levi & Jinglong Zhao, 2023. "Design and Analysis of Switchback Experiments," Management Science, INFORMS, vol. 69(7), pages 3759-3777, July.
    3. Jiafeng Chen, 2021. "Nonparametric Treatment Effect Identification in School Choice," Papers 2112.03872, arXiv.org, revised Oct 2023.
    4. Jinglong Zhao & Zijie Zhou, 2022. "Pigeonhole Design: Balancing Sequential Experiments from an Online Matching Perspective," Papers 2201.12936, arXiv.org, revised Oct 2023.
    5. Shan Huang & Chen Wang & Yuan Yuan & Jinglong Zhao & Jingjing Zhang, 2023. "Estimating Effects of Long-Term Treatments," Papers 2308.08152, arXiv.org.
    6. Jinglong Zhao, 2023. "Adaptive Neyman Allocation," Papers 2309.08808, arXiv.org, revised Sep 2023.
    7. Sven Resnjanskij & Jens Ruhose & Simon Wiederhold & Ludger Wößmann, 2021. "Mentoring Improves the Labor-Market Prospects of Highly Disadvantaged Adolescents," ifo Schnelldienst, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 74(02), pages 31-38, February.
    8. Alexandre Belloni & Victor Chernozhukov & Denis Chetverikov & Christian Hansen & Kengo Kato, 2018. "High-dimensional econometrics and regularized GMM," CeMMAP working papers CWP35/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    9. Dimitris Bertsimas & Agni Orfanoudaki & Rory B. Weiner, 2020. "Personalized treatment for coronary artery disease patients: a machine learning approach," Health Care Management Science, Springer, vol. 23(4), pages 482-506, December.
    10. Clément de Chaisemartin & Jaime Ramirez-Cuellar, 2024. "At What Level Should One Cluster Standard Errors in Paired and Small-Strata Experiments?," American Economic Journal: Applied Economics, American Economic Association, vol. 16(1), pages 193-212, January.
    11. Clément de Chaisemartin & Luc Behaghel, 2020. "Estimating the Effect of Treatments Allocated by Randomized Waiting Lists," Econometrica, Econometric Society, vol. 88(4), pages 1453-1477, July.
    12. Bruno Ferman & Cristine Pinto & Vitor Possebom, 2020. "Cherry Picking with Synthetic Controls," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 510-532, March.
    13. Bonesrønning, Hans & Finseraas, Henning & Hardoy, Ines & Iversen, Jon Marius Vaag & Nyhus, Ole Henning & Opheim, Vibeke & Salvanes, Kari Vea & Sandsør, Astrid Marie Jorde & Schøne, Pål, 2022. "Small-group instruction to improve student performance in mathematics in early grades: Results from a randomized field experiment," Journal of Public Economics, Elsevier, vol. 216(C).
    14. Peydró, José-Luis & Jiménez, Gabriel & Kenan, Huremovic & Moral-Benito, Enrique & Vega-Redondo, Fernando, 2020. "Production and financial networks in interplay: Crisis evidence from supplier-customer and credit registers," CEPR Discussion Papers 15277, C.E.P.R. Discussion Papers.
    15. Ruoxuan Xiong & Allison Koenecke & Michael Powell & Zhu Shen & Joshua T. Vogelstein & Susan Athey, 2021. "Federated Causal Inference in Heterogeneous Observational Data," Papers 2107.11732, arXiv.org, revised Apr 2023.
    16. Marie Bjørneby & Annette Alstadsæter & Kjetil Telle, 2018. "Collusive tax evasion by employers and employees. Evidence from a randomized fi eld experiment in Norway," Discussion Papers 891, Statistics Norway, Research Department.
    17. Konrad Menzel, 2021. "Structural Sieves," Papers 2112.01377, arXiv.org, revised Apr 2022.
    18. Susan Athey & Guido W. Imbens & Stefan Wager, 2018. "Approximate residual balancing: debiased inference of average treatment effects in high dimensions," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 80(4), pages 597-623, September.
    19. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2020. "Sampling‐Based versus Design‐Based Uncertainty in Regression Analysis," Econometrica, Econometric Society, vol. 88(1), pages 265-296, January.
    20. Andrés Elberg & Pedro M. Gardete & Rosario Macera & Carlos Noton, 2019. "Dynamic effects of price promotions: field evidence, consumer search, and supply-side implications," Quantitative Marketing and Economics (QME), Springer, vol. 17(1), pages 1-58, March.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2112.13495. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.