IDEAS home Printed from https://ideas.repec.org/p/cwl/cwldpp/2127r.html
   My bibliography  Save this paper

Experiment-as-Market: Incorporating Welfare into Randomized Controlled Trials

Author

Abstract

Randomized Controlled Trials (RCTs) enroll hundreds of millions of subjects and involve many human lives. To improve subjects' welfare, I propose a design of RCTs that I call Experiment-as-Market (EXAM). EXAM produces a Pareto efficient allocation of treatment assignment probabilities, is asymptotically incentive compatible for preference elicitation, and unbiasedly estimates any causal effect estimable with standard RCTs. I quantify these properties by applying EXAM to a water cleaning experiment in Kenya (Kremer et al., 2011). In this empirical setting, compared to standard RCTs, EXAM substantially improves subjects' predicted well-being while reaching similar treatment effect estimates with similar precision.

Suggested Citation

  • Yusuke Narita, 2018. "Experiment-as-Market: Incorporating Welfare into Randomized Controlled Trials," Cowles Foundation Discussion Papers 2127r, Cowles Foundation for Research in Economics, Yale University, revised May 2019.
    • Handle: RePEc:cwl:cwldpp:2127r
    as

    Download full text from publisher

    File URL: https://cowles.yale.edu/sites/default/files/files/pub/d21/d2127-r.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Jinyong Hahn & Keisuke Hirano & Dean Karlan, 2011. "Adaptive Experimental Design Using the Propensity Score," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 29(1), pages 96-108, January.
    2. Eduardo M Azevedo & Eric Budish, 2019. "Strategy-proofness in the Large," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 86(1), pages 81-116.
    3. James J. Heckman & Edward Vytlacil, 2005. "Structural Equations, Treatment Effects, and Econometric Policy Evaluation," Econometrica, Econometric Society, vol. 73(3), pages 669-738, May.
    4. James Berry & Greg Fischer & Raymond Guiteras, 2020. "Eliciting and Utilizing Willingness to Pay: Evidence from Field Trials in Northern Ghana," Journal of Political Economy, University of Chicago Press, vol. 128(4), pages 1436-1473.
    5. Atila Abdulkadiroğlu & Joshua D. Angrist & Yusuke Narita & Parag A. Pathak, 2017. "Research Design Meets Market Design: Using Centralized Assignment for Impact Evaluation," Econometrica, Econometric Society, vol. 85, pages 1373-1432, September.
    6. Toru Kitagawa & Aleksey Tetenov, 2018. "Who Should Be Treated? Empirical Welfare Maximization Methods for Treatment Choice," Econometrica, Econometric Society, vol. 86(2), pages 591-616, March.
    7. Tat Y. Chan & Barton H. Hamilton, 2006. "Learning, Private Information, and the Economic Evaluation of Randomized Experiments," Journal of Political Economy, University of Chicago Press, vol. 114(6), pages 997-1040, December.
    8. Ashraf, Nava & Karlan, Dean & Yin, Wesley, 2010. "Female Empowerment: Impact of a Commitment Savings Product in the Philippines," World Development, Elsevier, vol. 38(3), pages 333-344, March.
    9. Angrist, J.D. & Imbens, G.W., 1991. "Sources of Identifying Information in Evaluation Models," Harvard Institute of Economic Research Working Papers 1568, Harvard - Institute of Economic Research.
    10. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 61, pages 3895-3962, Elsevier.
    11. Nava Ashraf & Dean Karlan & Wesley Yin, 2006. "Tying Odysseus to the Mast: Evidence From a Commitment Savings Product in the Philippines," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 121(2), pages 635-672.
    12. Abadie, Alberto & Athey, Susan & Imbens, Guido W. & Wooldridge, Jeffrey M., 2017. "Sampling-Based vs. Design-Based Uncertainty in Regression Analysis," Research Papers 3349, Stanford University, Graduate School of Business.
    13. Sylvain Chassang & Gerard Padro I Miquel & Erik Snowberg, 2012. "Selective Trials: A Principal-Agent Approach to Randomized Controlled Experiments," American Economic Review, American Economic Association, vol. 102(4), pages 1279-1309, June.
    14. Jackson, Matthew O., 1992. "Incentive compatibility and competitive allocations," Economics Letters, Elsevier, vol. 40(3), pages 299-302, November.
    15. Train,Kenneth E., 2009. "Discrete Choice Methods with Simulation," Cambridge Books, Cambridge University Press, number 9780521766555, January.
    16. Michael Kremer & Jessica Leino & Edward Miguel & Alix Peterson Zwane, 2011. "Spring Cleaning: Rural Water Impacts, Valuation, and Property Rights Institutions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(1), pages 145-205.
    17. Nava Ashraf & James Berry & Jesse M. Shapiro, 2010. "Can Higher Prices Stimulate Product Use? Evidence from a Field Experiment in Zambia," American Economic Review, American Economic Association, vol. 100(5), pages 2383-2413, December.
    18. Christopher R. Walters, 2018. "The Demand for Effective Charter Schools," Journal of Political Economy, University of Chicago Press, vol. 126(6), pages 2179-2223.
    19. Hylland, Aanund & Zeckhauser, Richard, 1979. "The Efficient Allocation of Individuals to Positions," Journal of Political Economy, University of Chicago Press, vol. 87(2), pages 293-314, April.
    20. Johannes Haushofer & Jeremy Shapiro, 2016. "The Short-term Impact of Unconditional Cash Transfers to the Poor: ExperimentalEvidence from Kenya," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(4), pages 1973-2042.
    21. Eric Budish & Yeon-Koo Che & Fuhito Kojima & Paul Milgrom, 2013. "Designing Random Allocation Mechanisms: Theory and Applications," American Economic Review, American Economic Association, vol. 103(2), pages 585-623, April.
    22. Kasy, Maximilian, 2016. "Why Experimenters Might Not Always Want to Randomize, and What They Could Do Instead," Political Analysis, Cambridge University Press, vol. 24(3), pages 324-338, July.
    23. Yinghua He & Antonio Miralles & Marek Pycia & Jianye Yan, 2018. "A Pseudo-Market Approach to Allocation with Priorities," American Economic Journal: Microeconomics, American Economic Association, vol. 10(3), pages 272-314, August.
    24. Hu, Feifang & Rosenberger, William F., 2003. "Optimality, Variability, Power: Evaluating Response-Adaptive Randomization Procedures for Treatment Comparisons," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 671-678, January.
    25. S. A. Murphy, 2003. "Optimal dynamic treatment regimes," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 65(2), pages 331-355, May.
    26. Bogomolnaia, Anna & Moulin, Herve, 2001. "A New Solution to the Random Assignment Problem," Journal of Economic Theory, Elsevier, vol. 100(2), pages 295-328, October.
    27. Magne Mogstad & Alexander Torgovitsky, 2018. "Identification and Extrapolation of Causal Effects with Instrumental Variables," Annual Review of Economics, Annual Reviews, vol. 10(1), pages 577-613, August.
    28. Yusuki Narita, 2016. "(Non)Randomization: A Theory of Quasi-Experimental Evaluation of School Quality," Cowles Foundation Discussion Papers 2056R, Cowles Foundation for Research in Economics, Yale University, revised Aug 2017.
    29. Florencia Devoto & Esther Duflo & Pascaline Dupas & William Parienté & Vincent Pons, 2012. "Happiness on Tap: Piped Water Adoption in Urban Morocco," American Economic Journal: Economic Policy, American Economic Association, vol. 4(4), pages 68-99, November.
    30. Berry, Steven & Levinsohn, James & Pakes, Ariel, 1995. "Automobile Prices in Market Equilibrium," Econometrica, Econometric Society, vol. 63(4), pages 841-890, July.
    31. Jessica Cohen & Pascaline Dupas, 2010. "Free Distribution or Cost-Sharing? Evidence from a Randomized Malaria Prevention Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(1), pages 1-45.
    32. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Patrick Kline & Christopher Walters, 2019. "Audits as Evidence: Experiments, Ensembles, and Enforcement," Papers 1907.06622, arXiv.org, revised Jul 2019.
    2. Yusuke Narita, 2020. "A Theory of Quasi-Experimental Evaluation of School Quality," Working Papers 2020-085, Human Capital and Economic Opportunity Working Group.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yusuke Narita, 2018. "Toward an Ethical Experiment," Cowles Foundation Discussion Papers 2127, Cowles Foundation for Research in Economics, Yale University.
    2. Nikhil Agarwal & Eric Budish, 2021. "Market Design," NBER Working Papers 29367, National Bureau of Economic Research, Inc.
    3. Sylvain Chassang & Gerard Padro I Miquel & Erik Snowberg, 2012. "Selective Trials: A Principal-Agent Approach to Randomized Controlled Experiments," American Economic Review, American Economic Association, vol. 102(4), pages 1279-1309, June.
    4. Abhijit Banerjee & Sylvain Chassang & Erik Snowberg, 2016. "Decision Theoretic Approaches to Experiment Design and External Validity," NBER Working Papers 22167, National Bureau of Economic Research, Inc.
    5. Paulina Oliva & B. Kelsey Jack & Samuel Bell & Elizabeth Mettetal & Christopher Severen, 2020. "Technology Adoption under Uncertainty: Take-Up and Subsequent Investment in Zambia," The Review of Economics and Statistics, MIT Press, vol. 102(3), pages 617-632, July.
    6. Miralles, Antonio & Pycia, Marek, 2021. "Foundations of pseudomarkets: Walrasian equilibria for discrete resources," Journal of Economic Theory, Elsevier, vol. 196(C).
    7. Hashimoto, Tadashi, 2018. "The generalized random priority mechanism with budgets," Journal of Economic Theory, Elsevier, vol. 177(C), pages 708-733.
    8. Burchardi, Konrad B. & de Quidt, Jonathan & Gulesci, Selim & Lerva, Benedetta & Tripodi, Stefano, 2021. "Testing willingness to pay elicitation mechanisms in the field: Evidence from Uganda," Journal of Development Economics, Elsevier, vol. 152(C).
    9. Matthew Krupoff & Ahmed Mushfiq Mobarak & Alexander van Geen, 2020. "Evaluating Strategies to Reduce Arsenic Poisoning in South Asia: A View from the Social Sciences," Asian Development Review, MIT Press, vol. 37(2), pages 21-44, September.
    10. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2016. "Policy evaluation, randomized controlled trials, and external validity—A systematic review," Economics Letters, Elsevier, vol. 147(C), pages 51-54.
    11. Hoffmann, Bridget, 2018. "Do non-monetary prices target the poor? Evidence from a field experiment in India," Journal of Development Economics, Elsevier, vol. 133(C), pages 15-32.
    12. Pedro Carneiro & Sokbae Lee & Daniel Wilhelm, 2020. "Optimal data collection for randomized control trials [Microcredit impacts: Evidence from a randomized microcredit program placement experiment by Compartamos Banco]," The Econometrics Journal, Royal Economic Society, vol. 23(1), pages 1-31.
    13. Jörg Peters & Jörg Langbein & Gareth Roberts, 2018. "Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity," The World Bank Research Observer, World Bank, vol. 33(1), pages 34-64.
    14. Raymond P. Guiteras & B. Kelsey Jack, 2014. "Incentives, Selection and Productivity in Labor Markets: Evidence from Rural Malawi," NBER Working Papers 19825, National Bureau of Economic Research, Inc.
    15. Meredith, Jennifer & Robinson, Jonathan & Walker, Sarah & Wydick, Bruce, 2013. "Keeping the doctor away: Experimental evidence on investment in preventative health products," Journal of Development Economics, Elsevier, vol. 105(C), pages 196-210.
    16. Karthik Muralidharan & Mauricio Romero & Kaspar Wüthrich, 2019. "Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments," NBER Working Papers 26562, National Bureau of Economic Research, Inc.
    17. He, Yinghua & Li, Sanxi & Yan, Jianye, 2015. "Evaluating assignment without transfers: A market perspective," Economics Letters, Elsevier, vol. 133(C), pages 40-44.
    18. Echenique, Federico & Miralles, Antonio & Zhang, Jun, 2023. "Balanced equilibrium in pseudo-markets with endowments," Games and Economic Behavior, Elsevier, vol. 141(C), pages 428-443.
    19. James Berry & Greg Fischer & Raymond Guiteras, 2020. "Eliciting and Utilizing Willingness to Pay: Evidence from Field Trials in Northern Ghana," Journal of Political Economy, University of Chicago Press, vol. 128(4), pages 1436-1473.
    20. Whittington, Dale & Jeuland, Marc & Barker, Kate & Yuen, Yvonne, 2012. "Setting Priorities, Targeting Subsidies among Water, Sanitation, and Preventive Health Interventions in Developing Countries," World Development, Elsevier, vol. 40(8), pages 1546-1568.

    More about this item

    Keywords

    Research Ethics; Clinical Trial; Social Experiment; A/B Test; Market Design; Causal Inference; Development Economics; Spring Protection; Discrete Choice;
    All these keywords.

    JEL classification:

    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • D47 - Microeconomics - - Market Structure, Pricing, and Design - - - Market Design
    • O15 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Economic Development: Human Resources; Human Development; Income Distribution; Migration

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cwl:cwldpp:2127r. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Brittany Ladd (email available below). General contact details of provider: https://edirc.repec.org/data/cowleus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.