IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2011.08174.html

Some searches may not work properly. We apologize for the inconvenience.

   My bibliography  Save this paper

Policy design in experiments with unknown interference

Author

Listed:
  • Davide Viviano
  • Jess Rudder

Abstract

This paper studies experimental designs for estimation and inference on policies with spillover effects. Units are organized into a finite number of large clusters and interact in unknown ways within each cluster. First, we introduce a single-wave experiment that, by varying the randomization across cluster pairs, estimates the marginal effect of a change in treatment probabilities, taking spillover effects into account. Using the marginal effect, we propose a test for policy optimality. Second, we design a multiple-wave experiment to estimate welfare-maximizing treatment rules. We provide strong theoretical guarantees and an implementation in a large-scale field experiment.

Suggested Citation

  • Davide Viviano & Jess Rudder, 2020. "Policy design in experiments with unknown interference," Papers 2011.08174, arXiv.org, revised May 2024.
    • Handle: RePEc:arx:papers:2011.08174
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2011.08174
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Eckles Dean & Karrer Brian & Ugander Johan, 2017. "Design and Analysis of Experiments in Networks: Reducing Bias from Interference," Journal of Causal Inference, De Gruyter, vol. 5(1), pages 1-23, March.
    2. Dennis Egger & Johannes Haushofer & Edward Miguel & Paul Niehaus & Michael Walker, 2022. "General Equilibrium Effects of Cash Transfers: Experimental Evidence From Kenya," Econometrica, Econometric Society, vol. 90(6), pages 2603-2643, November.
    3. Timothy B. Armstrong & Shu Shen, 2013. "Inference on Optimal Treatment Assignments," Cowles Foundation Discussion Papers 1927RR, Cowles Foundation for Research in Economics, Yale University, revised Apr 2015.
    4. Charles F. Manski, 2013. "Identification of treatment response with social interactions," Econometrics Journal, Royal Economic Society, vol. 16(1), pages 1-23, February.
    5. Stoye, Jörg, 2009. "Minimax regret treatment choice with finite samples," Journal of Econometrics, Elsevier, vol. 151(1), pages 70-81, July.
    6. Isaiah Andrews & Toru Kitagawa & Adam McCloskey, 2024. "Inference on Winners," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 139(1), pages 305-358.
    7. Davide Viviano, 2020. "Experimental Design under Network Interference," Papers 2003.08421, arXiv.org, revised Jul 2022.
    8. Masahiro Kato & Yusuke Kaneko, 2020. "Off-Policy Evaluation of Bandit Algorithm from Dependent Samples under Batch Update Policy," Papers 2010.13554, arXiv.org.
    9. David Choi, 2017. "Estimation of Monotone Treatment Effects in Network Experiments," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 1147-1155, July.
    10. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey & James Robins, 2018. "Double/debiased machine learning for treatment and structural parameters," Econometrics Journal, Royal Economic Society, vol. 21(1), pages 1-68, February.
    11. Dehejia, Rajeev H., 2005. "Program evaluation as a decision problem," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 141-173.
    12. Debopam Bhattacharya & Pascaline Dupas & Shin Kanaya, 2024. "Demand and Welfare Analysis in Discrete Choice Models with Social Interactions," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 91(2), pages 748-784.
    13. Susan Athey & Dean Eckles & Guido W. Imbens, 2018. "Exact p-Values for Network Interference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(521), pages 230-240, January.
    14. Kasy, Maximilian, 2018. "Optimal taxation and insurance using machine learning — Sufficient statistics and beyond," Journal of Public Economics, Elsevier, vol. 167(C), pages 205-219.
    15. Eric Mbakop & Max Tabord‐Meehan, 2021. "Model Selection for Treatment Choice: Penalized Welfare Maximization," Econometrica, Econometric Society, vol. 89(2), pages 825-848, March.
    16. Maximilian Kasy & Anja Sautmann, 2021. "Adaptive Treatment Assignment in Experiments for Policy Choice," Econometrica, Econometric Society, vol. 89(1), pages 113-132, January.
    17. Toru Kitagawa & Aleksey Tetenov, 2018. "Who Should Be Treated? Empirical Welfare Maximization Methods for Treatment Choice," Econometrica, Econometric Society, vol. 86(2), pages 591-616, March.
    18. Karthik Muralidharan & Paul Niehaus & Sandip Sukhtankar, 2023. "General Equilibrium Effects of (Improving) Public Employment Programs: Experimental Evidence From India," Econometrica, Econometric Society, vol. 91(4), pages 1261-1295, July.
    19. Maximilian Kasy, 2016. "Partial Identification, Distributional Preferences, and the Welfare Ranking of Policies," The Review of Economics and Statistics, MIT Press, vol. 98(1), pages 111-131, March.
    20. Paul Goldsmith-Pinkham & Guido W. Imbens, 2013. "Social Networks and the Identification of Peer Effects," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 31(3), pages 253-264, July.
    21. Karthik Muralidharan & Paul Niehaus, 2017. "Experimentation at Scale," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 103-124, Fall.
    22. Bhattacharya, Debopam & Dupas, Pascaline, 2012. "Inferring welfare maximizing treatment assignment under budget constraints," Journal of Econometrics, Elsevier, vol. 167(1), pages 168-196.
    23. Athey, Susan & Imbens, Guido W., 2022. "Design-based analysis in Difference-In-Differences settings with staggered adoption," Journal of Econometrics, Elsevier, vol. 226(1), pages 62-79.
    24. Bhattacharya, Debopam, 2009. "Inferring Optimal Peer Assignment From Experimental Data," Journal of the American Statistical Association, American Statistical Association, vol. 104(486), pages 486-500.
    25. Charles F. Manski, 2004. "Statistical Treatment Rules for Heterogeneous Populations," Econometrica, Econometric Society, vol. 72(4), pages 1221-1246, July.
    26. Tetenov, Aleksey, 2012. "Statistical treatment choice based on asymmetric minimax regret criteria," Journal of Econometrics, Elsevier, vol. 166(1), pages 157-165.
    27. S. A. Murphy, 2003. "Optimal dynamic treatment regimes," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 65(2), pages 331-355, May.
    28. Elliott, Graham & Lieli, Robert P., 2013. "Predicting binary outcomes," Journal of Econometrics, Elsevier, vol. 174(1), pages 15-26.
    29. Guillaume Basse & Avi Feller, 2018. "Analyzing Two-Stage Experiments in the Presence of Interference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(521), pages 41-55, January.
    30. Debopam Bhattacharya & Pascaline Dupas & Shin Kanaya, 2013. "Estimating the Impact of Means-tested Subsidies under Treatment Externalities with Application to Anti-Malarial Bednets," Economics Series Working Papers 646, University of Oxford, Department of Economics.
    31. Xinkun Nie & Emma Brunskill & Stefan Wager, 2020. "Learning When-to-Treat Policies," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(533), pages 392-409, November.
    32. Jing Cai & Alain De Janvry & Elisabeth Sadoulet, 2015. "Social Networks and the Decision to Insure," American Economic Journal: Applied Economics, American Economic Association, vol. 7(2), pages 81-108, April.
    33. Xinran Li & Peng Ding & Qian Lin & Dawei Yang & Jun S. Liu, 2019. "Randomization Inference for Peer Effects," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(528), pages 1651-1664, October.
    34. Emmanuel Saez, 2001. "Using Elasticities to Derive Optimal Income Tax Rates," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 68(1), pages 205-229.
    35. Pascaline Dupas, 2014. "Short‐Run Subsidies and Long‐Run Adoption of New Health Products: Evidence From a Field Experiment," Econometrica, Econometric Society, vol. 82(1), pages 197-228, January.
    36. Susan Athey & Stefan Wager, 2021. "Policy Learning With Observational Data," Econometrica, Econometric Society, vol. 89(1), pages 133-161, January.
    37. Bryan S. Graham & Guido W. Imbens & Geert Ridder, 2010. "Measuring the Effects of Segregation in the Presence of Social Spillovers: A Nonparametric Approach," NBER Working Papers 16499, National Bureau of Economic Research, Inc.
    38. Max Tabord-Meehan, 2023. "Stratification Trees for Adaptive Randomisation in Randomised Controlled Trials," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(5), pages 2646-2673.
    39. Jason J Jones & Robert M Bond & Eytan Bakshy & Dean Eckles & James H Fowler, 2017. "Social influence and political mobilization: Further evidence from a randomized experiment in the 2012 U.S. presidential election," PLOS ONE, Public Library of Science, vol. 12(4), pages 1-9, April.
    40. Michael P. Leung, 2020. "Treatment and Spillover Effects Under Network Interference," The Review of Economics and Statistics, MIT Press, vol. 102(2), pages 368-380, May.
    41. Emily Breza & Arun G. Chandrasekhar & Tyler H. McCormick & Mengjie Pan, 2020. "Using Aggregated Relational Data to Feasibly Identify Network Structure without Network Data," American Economic Review, American Economic Association, vol. 110(8), pages 2454-2484, August.
    42. Hudgens, Michael G. & Halloran, M. Elizabeth, 2008. "Toward Causal Inference With Interference," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 832-842, June.
    43. Ibragimov, Rustam & Müller, Ulrich K., 2010. "t-Statistic Based Correlation and Heterogeneity Robust Inference," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(4), pages 453-468.
    44. Sarah Baird & J. Aislinn Bohren & Craig McIntosh & Berk Özler, 2018. "Optimal Design of Experiments in the Presence of Interference," The Review of Economics and Statistics, MIT Press, vol. 100(5), pages 844-860, December.
    45. Robert M. Bond & Christopher J. Fariss & Jason J. Jones & Adam D. I. Kramer & Cameron Marlow & Jaime E. Settle & James H. Fowler, 2012. "A 61-million-person experiment in social influence and political mobilization," Nature, Nature, vol. 489(7415), pages 295-298, September.
    46. Stoye, Jörg, 2012. "Minimax regret treatment choice with covariates or with limited validity of experiments," Journal of Econometrics, Elsevier, vol. 166(1), pages 138-156.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Davide Viviano, 2019. "Policy Targeting under Network Interference," Papers 1906.10258, arXiv.org, revised Apr 2024.
    2. Davide Viviano, 2020. "Experimental Design under Network Interference," Papers 2003.08421, arXiv.org, revised Jul 2022.
    3. Davide Viviano & Jelena Bradic, 2020. "Fair Policy Targeting," Papers 2005.12395, arXiv.org, revised Jun 2022.
    4. Yuehao Bai & Azeem M. Shaikh & Max Tabord-Meehan, 2024. "A Primer on the Analysis of Randomized Experiments and a Survey of some Recent Advances," Papers 2405.03910, arXiv.org.
    5. Kitagawa, Toru & Wang, Guanyi, 2023. "Who should get vaccinated? Individualized allocation of vaccines over SIR network," Journal of Econometrics, Elsevier, vol. 232(1), pages 109-131.
    6. Susan Athey & Stefan Wager, 2021. "Policy Learning With Observational Data," Econometrica, Econometric Society, vol. 89(1), pages 133-161, January.
    7. Shosei Sakaguchi, 2021. "Estimation of Optimal Dynamic Treatment Assignment Rules under Policy Constraints," Papers 2106.05031, arXiv.org, revised Aug 2024.
    8. Yuya Sasaki & Takuya Ura, 2020. "Welfare Analysis via Marginal Treatment Effects," Papers 2012.07624, arXiv.org.
    9. Toru Kitagawa & Guanyi Wang, 2021. "Who should get vaccinated? Individualized allocation of vaccines over SIR network," CeMMAP working papers CWP28/21, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    10. Eric Mbakop & Max Tabord‐Meehan, 2021. "Model Selection for Treatment Choice: Penalized Welfare Maximization," Econometrica, Econometric Society, vol. 89(2), pages 825-848, March.
    11. Davide Viviano & Lihua Lei & Guido Imbens & Brian Karrer & Okke Schrijvers & Liang Shi, 2023. "Causal clustering: design of cluster experiments under network interference," Papers 2310.14983, arXiv.org, revised Jan 2024.
    12. Chunrong Ai & Yue Fang & Haitian Xie, 2024. "Data-driven Policy Learning for a Continuous Treatment," Papers 2402.02535, arXiv.org.
    13. Toru Kitagawa & Sokbae Lee & Chen Qiu, 2022. "Treatment Choice with Nonlinear Regret," Papers 2205.08586, arXiv.org, revised Feb 2024.
    14. Kock, Anders Bredahl & Preinerstorfer, David & Veliyev, Bezirgen, 2023. "Treatment recommendation with distributional targets," Journal of Econometrics, Elsevier, vol. 234(2), pages 624-646.
    15. Debopam Bhattacharya & Pascaline Dupas & Shin Kanaya, 2013. "Estimating the Impact of Means-tested Subsidies under Treatment Externalities with Application to Anti-Malarial Bednets," CREATES Research Papers 2013-06, Department of Economics and Business Economics, Aarhus University.
    16. Toru Kitagawa & Weining Wang & Mengshan Xu, 2022. "Policy Choice in Time Series by Empirical Welfare Maximization," Papers 2205.03970, arXiv.org, revised Jun 2023.
    17. Anders Bredahl Kock & David Preinerstorfer, 2024. "Regularizing Discrimination in Optimal Policy Learning with Distributional Targets," Papers 2401.17909, arXiv.org.
    18. Toru Kitagawa & Guanyi Wang, 2020. "Who should get vaccinated? Individualized allocation of vaccines over SIR network," CeMMAP working papers CWP59/20, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    19. Michael P. Leung, 2020. "Treatment and Spillover Effects Under Network Interference," The Review of Economics and Statistics, MIT Press, vol. 102(2), pages 368-380, May.
    20. Kohei Yata, 2021. "Optimal Decision Rules Under Partial Identification," Papers 2111.04926, arXiv.org, revised Aug 2023.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2011.08174. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.