IDEAS home Printed from https://ideas.repec.org/p/hka/wpaper/2019-025.html
   My bibliography  Save this paper

Experiment-as-Market: Incorporating Welfare into Randomized Controlled Trials

Author

Listed:
  • Yusuke Narita

    (Massachusetts Institute of Technology)

Abstract

Randomized Controlled Trials (RCTs) enroll hundreds of millions of subjects and involve many human lives. To improve subjects' welfare, I propose a design of RCTs that I call Experiment-as-Market (EXAM). EXAM produces a Pareto efficient allocation of treatment assignment probabilities, is asymptotically incentive compatible for preference elicitation, and unbiasedly estimates any causal effect estimable with standard RCTs. I quantify these properties by applying EXAM to a water cleaning experiment in Kenya (Kremer et al., 2011). In this empirical setting, compared to standard RCTs, EXAM substantially improves subjects' predicted well-being while reaching similar treatment effect estimates with similar precision.

Suggested Citation

  • Yusuke Narita, 2019. "Experiment-as-Market: Incorporating Welfare into Randomized Controlled Trials," Working Papers 2019-025, Human Capital and Economic Opportunity Working Group.
    • Handle: RePEc:hka:wpaper:2019-025
    Note: MIP
    as

    Download full text from publisher

    File URL: http://humcap.uchicago.edu/RePEc/hka/wpaper/Narita_2019_experiment-as-market.pdf
    File Function: First version, January 10, 2019
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Jinyong Hahn & Keisuke Hirano & Dean Karlan, 2011. "Adaptive Experimental Design Using the Propensity Score," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 29(1), pages 96-108, January.
    2. Yinghua He & Antonio Miralles & Marek Pycia & Jianye Yan, 2018. "A Pseudo-Market Approach to Allocation with Priorities," American Economic Journal: Microeconomics, American Economic Association, vol. 10(3), pages 272-314, August.
    3. Hu, Feifang & Rosenberger, William F., 2003. "Optimality, Variability, Power: Evaluating Response-Adaptive Randomization Procedures for Treatment Comparisons," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 671-678, January.
    4. Eduardo M Azevedo & Eric Budish, 2019. "Strategy-proofness in the Large," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 86(1), pages 81-116.
    5. James J. Heckman & Edward Vytlacil, 2005. "Structural Equations, Treatment Effects, and Econometric Policy Evaluation," Econometrica, Econometric Society, vol. 73(3), pages 669-738, May.
    6. James Berry & Greg Fischer & Raymond Guiteras, 2020. "Eliciting and Utilizing Willingness to Pay: Evidence from Field Trials in Northern Ghana," Journal of Political Economy, University of Chicago Press, vol. 128(4), pages 1436-1473.
    7. Atila Abdulkadiroğlu & Joshua D. Angrist & Yusuke Narita & Parag A. Pathak, 2017. "Research Design Meets Market Design: Using Centralized Assignment for Impact Evaluation," Econometrica, Econometric Society, vol. 85, pages 1373-1432, September.
    8. Toru Kitagawa & Aleksey Tetenov, 2018. "Who Should Be Treated? Empirical Welfare Maximization Methods for Treatment Choice," Econometrica, Econometric Society, vol. 86(2), pages 591-616, March.
    9. Kasy, Maximilian, 2016. "Why Experimenters Might Not Always Want to Randomize, and What They Could Do Instead," Political Analysis, Cambridge University Press, vol. 24(3), pages 324-338, July.
    10. S. A. Murphy, 2003. "Optimal dynamic treatment regimes," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 65(2), pages 331-355, May.
    11. Tat Y. Chan & Barton H. Hamilton, 2006. "Learning, Private Information, and the Economic Evaluation of Randomized Experiments," Journal of Political Economy, University of Chicago Press, vol. 114(6), pages 997-1040, December.
    12. Ashraf, Nava & Karlan, Dean & Yin, Wesley, 2010. "Female Empowerment: Impact of a Commitment Savings Product in the Philippines," World Development, Elsevier, vol. 38(3), pages 333-344, March.
    13. Angrist, J.D. & Imbens, G.W., 1991. "Sources of Identifying Information in Evaluation Models," Harvard Institute of Economic Research Working Papers 1568, Harvard - Institute of Economic Research.
    14. Bogomolnaia, Anna & Moulin, Herve, 2001. "A New Solution to the Random Assignment Problem," Journal of Economic Theory, Elsevier, vol. 100(2), pages 295-328, October.
    15. Sylvain Chassang & Gerard Padro I Miquel & Erik Snowberg, 2012. "Selective Trials: A Principal-Agent Approach to Randomized Controlled Experiments," American Economic Review, American Economic Association, vol. 102(4), pages 1279-1309, June.
    16. Magne Mogstad & Alexander Torgovitsky, 2018. "Identification and Extrapolation of Causal Effects with Instrumental Variables," Annual Review of Economics, Annual Reviews, vol. 10(1), pages 577-613, August.
    17. Angrist, J.D. & Imbens, G.W., 1991. "Sources of Identifying Information in Evaluation Models," Harvard Institute of Economic Research Working Papers 1568, Harvard - Institute of Economic Research.
    18. Michael Kremer & Jessica Leino & Edward Miguel & Alix Peterson Zwane, 2011. "Spring Cleaning: Rural Water Impacts, Valuation, and Property Rights Institutions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(1), pages 145-205.
    19. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 61, pages 3895-3962, Elsevier.
    20. Yusuki Narita, 2016. "(Non)Randomization: A Theory of Quasi-Experimental Evaluation of School Quality," Cowles Foundation Discussion Papers 2056R, Cowles Foundation for Research in Economics, Yale University, revised Aug 2017.
    21. Florencia Devoto & Esther Duflo & Pascaline Dupas & William Parienté & Vincent Pons, 2012. "Happiness on Tap: Piped Water Adoption in Urban Morocco," American Economic Journal: Economic Policy, American Economic Association, vol. 4(4), pages 68-99, November.
    22. Nava Ashraf & Dean Karlan & Wesley Yin, 2006. "Tying Odysseus to the Mast: Evidence From a Commitment Savings Product in the Philippines," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 121(2), pages 635-672.
    23. Abadie, Alberto & Athey, Susan & Imbens, Guido W. & Wooldridge, Jeffrey M., 2017. "Sampling-Based vs. Design-Based Uncertainty in Regression Analysis," Research Papers 3349, Stanford University, Graduate School of Business.
    24. Train,Kenneth E., 2009. "Discrete Choice Methods with Simulation," Cambridge Books, Cambridge University Press, number 9780521747387, October.
    25. Jackson, Matthew O., 1992. "Incentive compatibility and competitive allocations," Economics Letters, Elsevier, vol. 40(3), pages 299-302, November.
    26. Nava Ashraf & James Berry & Jesse M. Shapiro, 2010. "Can Higher Prices Stimulate Product Use? Evidence from a Field Experiment in Zambia," American Economic Review, American Economic Association, vol. 100(5), pages 2383-2413, December.
    27. Christopher R. Walters, 2018. "The Demand for Effective Charter Schools," Journal of Political Economy, University of Chicago Press, vol. 126(6), pages 2179-2223.
    28. Berry, Steven & Levinsohn, James & Pakes, Ariel, 1995. "Automobile Prices in Market Equilibrium," Econometrica, Econometric Society, vol. 63(4), pages 841-890, July.
    29. Jessica Cohen & Pascaline Dupas, 2010. "Free Distribution or Cost-Sharing? Evidence from a Randomized Malaria Prevention Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(1), pages 1-45.
    30. Hylland, Aanund & Zeckhauser, Richard, 1979. "The Efficient Allocation of Individuals to Positions," Journal of Political Economy, University of Chicago Press, vol. 87(2), pages 293-314, April.
    31. Johannes Haushofer & Jeremy Shapiro, 2016. "The Short-term Impact of Unconditional Cash Transfers to the Poor: ExperimentalEvidence from Kenya," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(4), pages 1973-2042.
    32. Eric Budish & Yeon-Koo Che & Fuhito Kojima & Paul Milgrom, 2013. "Designing Random Allocation Mechanisms: Theory and Applications," American Economic Review, American Economic Association, vol. 103(2), pages 585-623, April.
    33. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yusuke Narita, 2020. "A Theory of Quasi-Experimental Evaluation of School Quality," Working Papers 2020-085, Human Capital and Economic Opportunity Working Group.
    2. Kline, Patrick & Walters, Christopher, 2019. "Audits as Evidence: Experiments, Ensembles, and Enforcement," Institute for Research on Labor and Employment, Working Paper Series qt3z72m9kn, Institute of Industrial Relations, UC Berkeley.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yusuke Narita, 2018. "Toward an Ethical Experiment," Cowles Foundation Discussion Papers 2127, Cowles Foundation for Research in Economics, Yale University.
    2. Nikhil Agarwal & Eric Budish, 2021. "Market Design," NBER Working Papers 29367, National Bureau of Economic Research, Inc.
    3. Sylvain Chassang & Gerard Padro I Miquel & Erik Snowberg, 2012. "Selective Trials: A Principal-Agent Approach to Randomized Controlled Experiments," American Economic Review, American Economic Association, vol. 102(4), pages 1279-1309, June.
    4. Abhijit Banerjee & Sylvain Chassang & Erik Snowberg, 2016. "Decision Theoretic Approaches to Experiment Design and External Validity," NBER Working Papers 22167, National Bureau of Economic Research, Inc.
    5. Miralles, Antonio & Pycia, Marek, 2021. "Foundations of pseudomarkets: Walrasian equilibria for discrete resources," Journal of Economic Theory, Elsevier, vol. 196(C).
    6. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2016. "Policy evaluation, randomized controlled trials, and external validity—A systematic review," Economics Letters, Elsevier, vol. 147(C), pages 51-54.
    7. Paulina Oliva & B. Kelsey Jack & Samuel Bell & Elizabeth Mettetal & Christopher Severen, 2020. "Technology Adoption under Uncertainty: Take-Up and Subsequent Investment in Zambia," The Review of Economics and Statistics, MIT Press, vol. 102(3), pages 617-632, July.
    8. Hashimoto, Tadashi, 2018. "The generalized random priority mechanism with budgets," Journal of Economic Theory, Elsevier, vol. 177(C), pages 708-733.
    9. Burchardi, Konrad B. & de Quidt, Jonathan & Gulesci, Selim & Lerva, Benedetta & Tripodi, Stefano, 2021. "Testing willingness to pay elicitation mechanisms in the field: Evidence from Uganda," Journal of Development Economics, Elsevier, vol. 152(C).
    10. Matthew Krupoff & Ahmed Mushfiq Mobarak & Alexander van Geen, 2020. "Evaluating Strategies to Reduce Arsenic Poisoning in South Asia: A View from the Social Sciences," Asian Development Review, MIT Press, vol. 37(2), pages 21-44, September.
    11. Pedro Carneiro & Sokbae Lee & Daniel Wilhelm, 2020. "Optimal data collection for randomized control trials," The Econometrics Journal, Royal Economic Society, vol. 23(1), pages 1-31.
    12. Hoffmann, Bridget, 2018. "Do non-monetary prices target the poor? Evidence from a field experiment in India," Journal of Development Economics, Elsevier, vol. 133(C), pages 15-32.
    13. Aufenanger, Tobias, 2017. "Machine learning to improve experimental design," FAU Discussion Papers in Economics 16/2017, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics, revised 2017.
    14. Jörg Peters & Jörg Langbein & Gareth Roberts, 2018. "Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity," The World Bank Research Observer, World Bank, vol. 33(1), pages 34-64.
    15. Shende, Priyanka & Purohit, Manish, 2023. "Strategy-proof and envy-free mechanisms for house allocation," Journal of Economic Theory, Elsevier, vol. 213(C).
    16. Raymond P. Guiteras & B. Kelsey Jack, 2014. "Incentives, Selection and Productivity in Labor Markets: Evidence from Rural Malawi," NBER Working Papers 19825, National Bureau of Economic Research, Inc.
    17. Meredith, Jennifer & Robinson, Jonathan & Walker, Sarah & Wydick, Bruce, 2013. "Keeping the doctor away: Experimental evidence on investment in preventative health products," Journal of Development Economics, Elsevier, vol. 105(C), pages 196-210.
    18. Karthik Muralidharan & Mauricio Romero & Kaspar Wüthrich, 2019. "Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments," NBER Working Papers 26562, National Bureau of Economic Research, Inc.
    19. Grimm, Michael & Hartwig, Renate, 2018. "Unblurring the Market for Vision Correction: A Willingness to Pay Experiment in Rural Burkina Faso," IZA Discussion Papers 11929, Institute of Labor Economics (IZA).
    20. James Berry & Greg Fischer & Raymond Guiteras, 2020. "Eliciting and Utilizing Willingness to Pay: Evidence from Field Trials in Northern Ghana," Journal of Political Economy, University of Chicago Press, vol. 128(4), pages 1436-1473.

    More about this item

    Keywords

    clinical trial; social experiments; A/B test; market design; competitive equilibrium from equal income; Pareto efficiency; causal inference; development economics;
    All these keywords.

    JEL classification:

    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • D47 - Microeconomics - - Market Structure, Pricing, and Design - - - Market Design
    • O15 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Economic Development: Human Resources; Human Development; Income Distribution; Migration

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hka:wpaper:2019-025. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Jennifer Pachon (email available below). General contact details of provider: https://edirc.repec.org/data/mfichus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.