IDEAS home Printed from https://ideas.repec.org/p/cpr/ceprdp/8003.html
   My bibliography  Save this paper

Selective Trials: A Principal-Agent Approach to Randomized Controlled Experiments

Author

Listed:
  • Chassang, Sylvain
  • Padró i Miquel, Gerard
  • Snowberg, Erik

Abstract

We study the design of randomized controlled experiments in environments where outcomes are significantly affected by unobserved effort decisions taken by the subjects (agents). While standard randomized controlled trials (RCTs) are internally consistent, the unobservability of effort provision compromises external validity. We approach trial design as a principal-agent problem and show that natural extensions of RCTs--which we call selective trials--can help improve the external validity of experiments. In particular, selective trials can disentangle the effects of treatment, effort, and the interaction of treatment and effort. Moreover, they can help experimenters identify when measured treatment effects are affected by erroneous beliefs and inappropriate effort provision.

Suggested Citation

  • Chassang, Sylvain & Padró i Miquel, Gerard & Snowberg, Erik, 2010. "Selective Trials: A Principal-Agent Approach to Randomized Controlled Experiments," CEPR Discussion Papers 8003, C.E.P.R. Discussion Papers.
  • Handle: RePEc:cpr:ceprdp:8003
    as

    Download full text from publisher

    File URL: http://www.cepr.org/active/publications/discussion_papers/dp.php?dpno=8003
    Download Restriction: CEPR Discussion Papers are free to download for our researchers, subscribers and members. If you fall into one of these categories but have trouble downloading our papers, please contact us at subscribers@cepr.org

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Esther Duflo & Rema Hanna, 2005. "Monitoring Works: Getting Teachers to Come to School," Working Papers id:301, eSocialSciences.
    2. Esther Duflo & Michael Kremer & Jonathan Robinson, 2008. "How High Are Rates of Return to Fertilizer? Evidence from Field Experiments in Kenya," American Economic Review, American Economic Association, vol. 98(2), pages 482-488, May.
    3. Jessica Cohen & Pascaline Dupas, 2008. "Free Distribution or Cost-Sharing? Evidence from a Malaria Prevention Experiment," NBER Working Papers 14406, National Bureau of Economic Research, Inc.
    4. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    5. Pagan,Adrian & Ullah,Aman, 1999. "Nonparametric Econometrics," Cambridge Books, Cambridge University Press, number 9780521355643.
    6. Emir Kamenica & Matthew Gentzkow, 2011. "Bayesian Persuasion," American Economic Review, American Economic Association, vol. 101(6), pages 2590-2615, October.
    7. Michael Kremer & Edward Miguel & Rebecca Thornton, 2009. "Incentives to Learn," The Review of Economics and Statistics, MIT Press, vol. 91(3), pages 437-456, August.
    8. James J. Heckman & Jeffrey Smith & Nancy Clements, 1997. "Making The Most Out Of Programme Evaluations and Social Experiments: Accounting For Heterogeneity in Programme Impacts," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 487-535.
    9. Tomas Philipson & Larry V. Hedges, 1998. "Subject Evaluation in Social Experiments," Econometrica, Econometric Society, vol. 66(2), pages 381-408, March.
    10. Sendhil Mullainathan & Joshua Schwartzstein & Andrei Shleifer, 2008. "Coarse Thinking and Persuasion," The Quarterly Journal of Economics, Oxford University Press, vol. 123(2), pages 577-619.
    11. Pascaline Dupas, 2014. "Short‐Run Subsidies and Long‐Run Adoption of New Health Products: Evidence From a Field Experiment," Econometrica, Econometric Society, vol. 82(1), pages 197-228, January.
    12. Michael Kremer & Edward Miguel, 2007. "The Illusion of Sustainability," The Quarterly Journal of Economics, Oxford University Press, vol. 122(3), pages 1007-1065.
    13. Joshua Angrist & Eric Bettinger & Erik Bloom & Elizabeth King & Michael Kremer, 2002. "Vouchers for Private Schooling in Colombia: Evidence from a Randomized Natural Experiment," American Economic Review, American Economic Association, vol. 92(5), pages 1535-1558, December.
    14. Bikhchandani, Sushil & Hirshleifer, David & Welch, Ivo, 1992. "A Theory of Fads, Fashion, Custom, and Cultural Change in Informational Cascades," Journal of Political Economy, University of Chicago Press, vol. 100(5), pages 992-1026, October.
    15. Dean Karlan & Jonathan Zinman, 2009. "Observing Unobservables: Identifying Information Asymmetries With a Consumer Credit Field Experiment," Econometrica, Econometric Society, vol. 77(6), pages 1993-2008, November.
    16. Fischer, Gregory & Berry, James & Guiteras, Raymond, 2012. "Eliciting and utilizing willingness to pay: evidence from field trials in Northern Ghana," LSE Research Online Documents on Economics 47913, London School of Economics and Political Science, LSE Library.
    17. Nava Ashraf & Dean Karlan & Wesley Yin, 2006. "Tying Odysseus to the Mast: Evidence From a Commitment Savings Product in the Philippines," The Quarterly Journal of Economics, Oxford University Press, vol. 121(2), pages 635-672.
    18. Pascaline Dupas, 2011. "Do Teenagers Respond to HIV Risk Information? Evidence from a Field Experiment in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 3(1), pages 1-34, January.
    19. Abhijit V. Banerjee, 1992. "A Simple Model of Herd Behavior," The Quarterly Journal of Economics, Oxford University Press, vol. 107(3), pages 797-817.
    20. Paul Milgrom & Ilya Segal, 2002. "Envelope Theorems for Arbitrary Choice Sets," Econometrica, Econometric Society, vol. 70(2), pages 583-601, March.
    21. Nava Ashraf & James Berry & Jesse M. Shapiro, 2010. "Can Higher Prices Stimulate Product Use? Evidence from a Field Experiment in Zambia," American Economic Review, American Economic Association, vol. 100(5), pages 2383-2413, December.
    22. Edward Miguel & Michael Kremer, 2004. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities," Econometrica, Econometric Society, vol. 72(1), pages 159-217, January.
    23. Jessica Cohen & Pascaline Dupas, 2010. "Free Distribution or Cost-Sharing? Evidence from a Randomized Malaria Prevention Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 125(1), pages 1-45.
    24. Malani, Anup, 2008. "Patient enrollment in medical trials: Selection bias in a randomized experiment," Journal of Econometrics, Elsevier, vol. 144(2), pages 341-351, June.
    25. Paul Schultz, T., 2004. "School subsidies for the poor: evaluating the Mexican Progresa poverty program," Journal of Development Economics, Elsevier, vol. 74(1), pages 199-250, June.
    26. Paul Gertler, 2004. "Do Conditional Cash Transfers Improve Child Health? Evidence from PROGRESA's Control Randomized Experiment," American Economic Review, American Economic Association, vol. 94(2), pages 336-341, May.
    27. A. D. Roy, 1951. "Some Thoughts On The Distribution Of Earnings," Oxford Economic Papers, Oxford University Press, vol. 3(2), pages 135-146.
    28. repec:feb:natura:0004 is not listed on IDEAS
    29. Rebecca L. Thornton, 2008. "The Demand for, and Impact of, Learning HIV Status," American Economic Review, American Economic Association, vol. 98(5), pages 1829-1863, December.
    30. Anup Malani, 2006. "Identifying Placebo Effects with Data from Clinical Trials," Journal of Political Economy, University of Chicago Press, vol. 114(2), pages 236-256, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Cristina Corduneanu-Huci & Michael T. Dorsch & Paul Maarek, 2017. "Learning to constrain: Political competition and randomized controlled trials in development," THEMA Working Papers 2017-24, THEMA (THéorie Economique, Modélisation et Applications), Université de Cergy-Pontoise.
    2. Fischer, Gregory & Berry, James & Guiteras, Raymond, 2012. "Eliciting and utilizing willingness to pay: evidence from field trials in Northern Ghana," LSE Research Online Documents on Economics 47913, London School of Economics and Political Science, LSE Library.
    3. Abhijit Banerjee & Sylvain Chassang & Erik Snowberg, 2016. "Decision Theoretic Approaches to Experiment Design and External Validity," NBER Working Papers 22167, National Bureau of Economic Research, Inc.
    4. Rema Hanna & Esther Duflo & Michael Greenstone, 2016. "Up in Smoke: The Influence of Household Behavior on the Long-Run Impact of Improved Cooking Stoves," American Economic Journal: Economic Policy, American Economic Association, vol. 8(1), pages 80-114, February.
    5. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2017. "Generalization in the Tropics: Development policy, randomized controlled trials, and external validity," Ruhr Economic Papers 716, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    6. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2016. "Policy evaluation, randomized controlled trials, and external validity—A systematic review," Economics Letters, Elsevier, vol. 147(C), pages 51-54.
    7. Gharad Bryan & Dean Karlan & Jonathan Zinman, 2012. "You Can Pick Your Friends, But You Need to Watch Them: Loan Screening and Enforcement in a Referrals Field Experiment," Working Papers 1009, Economic Growth Center, Yale University.
    8. Anup Malani & Tomas J. Philipson, 2011. "Can Medical Progress be Sustained? Implications of the Link Between Development and Output Markets," NBER Working Papers 17011, National Bureau of Economic Research, Inc.
    9. repec:aea:jecper:v:31:y:2017:i:4:p:73-102 is not listed on IDEAS
    10. Amanda Kowalski, 2016. "Doing more when you're running LATE: Applying marginal treatment effect methods to examine treatment effect heterogeneity in experiments," Artefactual Field Experiments 00560, The Field Experiments Website.
    11. B. Kelsey Jack & Paulina Oliva & Christopher Severen & Elizabeth Walker & Samuel Bell, 2015. "Technology Adoption Under Uncertainty: Take-Up and Subsequent Investment in Zambia," NBER Working Papers 21414, National Bureau of Economic Research, Inc.
    12. Fitzsimons, Emla & Malde, Bansi & Mesnard, Alice & Vera-Hernández, Marcos, 2012. "Household Responses to Information on Child Nutrition: Experimental Evidence from Malawi," CEPR Discussion Papers 8915, C.E.P.R. Discussion Papers.
    13. Amanda E. Kowalski, 2016. "Doing More When You're Running LATE: Applying Marginal Treatment Effect Methods to Examine Treatment Effect Heterogeneity in Experiments for the Young and Privately Insured?," Cowles Foundation Discussion Papers 2045, Cowles Foundation for Research in Economics, Yale University.
    14. Eva Vivalt, 2015. "Heterogeneous Treatment Effects in Impact Evaluation," American Economic Review, American Economic Association, vol. 105(5), pages 467-470, May.
    15. Hunt Allcott, 2012. "Site Selection Bias in Program Evaluation," NBER Working Papers 18373, National Bureau of Economic Research, Inc.
    16. Abhijit Banerjee & Rukmini Banerji & James Berry & Esther Duflo & Harini Kannan & Shobhini Mukerji & Marc Shotland & Michael Walton, 2017. "From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 73-102, Fall.
    17. Abhijit Banerjee & Rukmini Banerji & James Berry & Esther Duflo & Harini Kannan, 2016. "From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application," Working Papers id:11516, eSocialSciences.
    18. Davis, Alexander L. & Krishnamurti, Tamar, 2013. "The problems and solutions of predicting participation in energy efficiency programs," Applied Energy, Elsevier, vol. 111(C), pages 277-287.
    19. Gani Aldashev & Georg Kirchsteiger & Alexander Sebald, 2012. "Assignment procedure biases in randomized policy experiments," Carlo Alberto Notebooks 292, Collegio Carlo Alberto.
    20. Juan Ortner & Sylvain Chassang, 2014. "Making Collusion Hard: Asymmetric Information as a Counter-Corruption Measure," Working Papers 064-2014, Princeton University, Department of Economics, Econometric Research Program..
    21. Raymond P. Guiteras & B. Kelsey Jack, 2014. "Incentives, Selection and Productivity in Labor Markets: Evidence from Rural Malawi," NBER Working Papers 19825, National Bureau of Economic Research, Inc.
    22. B. Kelsey Jack, 2013. "Private Information and the Allocation of Land Use Subsidies in Malawi," American Economic Journal: Applied Economics, American Economic Association, vol. 5(3), pages 113-135, July.

    More about this item

    Keywords

    blind trials; compliance; heterogeneous beliefs; incentivized trials; marginal treatment e ects; mechanism design; randomized controlled trials; selection; selective trials;

    JEL classification:

    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • D82 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Asymmetric and Private Information; Mechanism Design
    • O12 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Microeconomic Analyses of Economic Development

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cpr:ceprdp:8003. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.