IDEAS home Printed from https://ideas.repec.org/a/aea/aecrev/v102y2012i4p1279-1309.html
   My bibliography  Save this article

Selective Trials: A Principal-Agent Approach to Randomized Controlled Experiments

Author

Listed:
  • Sylvain Chassang
  • Gerard Padro I Miquel
  • Erik Snowberg

Abstract

We study the design of randomized controlled experiments when outcomes are significantly affected by experimental subjects' unobserved effort expenditure. While standard randomized controlled trials (RCTs) are internally consistent, the unobservability of effort compromises external validity. We approach trial design as a principal-agent problem and show that natural extensions of RCTs--which we call selective trials--can help improve external validity. In particular, selective trials can disentangle the effects of treatment, effort, and the interaction of treatment and effort. Moreover, they can help identify when treatment effects are affected by erroneous beliefs and inappropriate effort expenditure. (JEL C90, D82)

Suggested Citation

  • Sylvain Chassang & Gerard Padro I Miquel & Erik Snowberg, 2012. "Selective Trials: A Principal-Agent Approach to Randomized Controlled Experiments," American Economic Review, American Economic Association, vol. 102(4), pages 1279-1309, June.
  • Handle: RePEc:aea:aecrev:v:102:y:2012:i:4:p:1279-1309
    as

    Download full text from publisher

    File URL: http://www.aeaweb.org/articles.php?doi=10.1257/aer.102.4.1279
    Download Restriction: Access to full text is restricted to AEA members and institutional subscribers.

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Paul Schultz, T., 2004. "School subsidies for the poor: evaluating the Mexican Progresa poverty program," Journal of Development Economics, Elsevier, vol. 74(1), pages 199-250, June.
    2. Esther Duflo & Rema Hanna, 2005. "Monitoring Works: Getting Teachers to Come to School," Working Papers id:301, eSocialSciences.
    3. Sendhil Mullainathan & Joshua Schwartzstein & Andrei Shleifer, 2008. "Coarse Thinking and Persuasion," The Quarterly Journal of Economics, Oxford University Press, vol. 123(2), pages 577-619.
    4. Berry, James & Fischer, Greg & Guiteras, Raymond, 2015. "Eliciting and Utilizing Willingness to Pay: Evidence from Field Trials in Northern Ghana," CEPR Discussion Papers 10703, C.E.P.R. Discussion Papers.
    5. Paul Milgrom & Ilya Segal, 2002. "Envelope Theorems for Arbitrary Choice Sets," Econometrica, Econometric Society, vol. 70(2), pages 583-601, March.
    6. Pascaline Dupas, 2014. "Short‐Run Subsidies and Long‐Run Adoption of New Health Products: Evidence From a Field Experiment," Econometrica, Econometric Society, vol. 82(1), pages 197-228, January.
    7. Esther Duflo & Michael Kremer & Jonathan Robinson, 2008. "How High Are Rates of Return to Fertilizer? Evidence from Field Experiments in Kenya," American Economic Review, American Economic Association, vol. 98(2), pages 482-488, May.
    8. Joshua Angrist & Eric Bettinger & Erik Bloom & Elizabeth King & Michael Kremer, 2002. "Vouchers for Private Schooling in Colombia: Evidence from a Randomized Natural Experiment," American Economic Review, American Economic Association, vol. 92(5), pages 1535-1558, December.
    9. Jessica Cohen & Pascaline Dupas, 2008. "Free Distribution or Cost-Sharing? Evidence from a Malaria Prevention Experiment," NBER Working Papers 14406, National Bureau of Economic Research, Inc.
    10. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    11. Michael Kremer & Edward Miguel, 2007. "The Illusion of Sustainability," The Quarterly Journal of Economics, Oxford University Press, vol. 122(3), pages 1007-1065.
    12. Emir Kamenica & Matthew Gentzkow, 2011. "Bayesian Persuasion," American Economic Review, American Economic Association, vol. 101(6), pages 2590-2615, October.
    13. Dean Karlan & Jonathan Zinman, 2009. "Observing Unobservables: Identifying Information Asymmetries With a Consumer Credit Field Experiment," Econometrica, Econometric Society, vol. 77(6), pages 1993-2008, November.
    14. Pagan,Adrian & Ullah,Aman, 1999. "Nonparametric Econometrics," Cambridge Books, Cambridge University Press, number 9780521355643.
    15. Pascaline Dupas, 2011. "Do Teenagers Respond to HIV Risk Information? Evidence from a Field Experiment in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 3(1), pages 1-34, January.
    16. Nava Ashraf & Dean Karlan & Wesley Yin, 2006. "Tying Odysseus to the Mast: Evidence From a Commitment Savings Product in the Philippines," The Quarterly Journal of Economics, Oxford University Press, vol. 121(2), pages 635-672.
    17. Paul Gertler, 2004. "Do Conditional Cash Transfers Improve Child Health? Evidence from PROGRESA's Control Randomized Experiment," American Economic Review, American Economic Association, vol. 94(2), pages 336-341, May.
    18. Nava Ashraf & James Berry & Jesse M. Shapiro, 2010. "Can Higher Prices Stimulate Product Use? Evidence from a Field Experiment in Zambia," American Economic Review, American Economic Association, vol. 100(5), pages 2383-2413, December.
    19. Bikhchandani, Sushil & Hirshleifer, David & Welch, Ivo, 1992. "A Theory of Fads, Fashion, Custom, and Cultural Change in Informational Cascades," Journal of Political Economy, University of Chicago Press, vol. 100(5), pages 992-1026, October.
    20. A. D. Roy, 1951. "Some Thoughts On The Distribution Of Earnings," Oxford Economic Papers, Oxford University Press, vol. 3(2), pages 135-146.
    21. Michael Kremer & Edward Miguel & Rebecca Thornton, 2009. "Incentives to Learn," The Review of Economics and Statistics, MIT Press, vol. 91(3), pages 437-456, August.
    22. Fischer, Gregory & Berry, James & Guiteras, Raymond, 2012. "Eliciting and utilizing willingness to pay: evidence from field trials in Northern Ghana," LSE Research Online Documents on Economics 47913, London School of Economics and Political Science, LSE Library.
    23. repec:feb:natura:0004 is not listed on IDEAS
    24. James J. Heckman & Jeffrey Smith & Nancy Clements, 1997. "Making The Most Out Of Programme Evaluations and Social Experiments: Accounting For Heterogeneity in Programme Impacts," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 487-535.
    25. Rebecca L. Thornton, 2008. "The Demand for, and Impact of, Learning HIV Status," American Economic Review, American Economic Association, vol. 98(5), pages 1829-1863, December.
    26. Edward Miguel & Michael Kremer, 2004. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities," Econometrica, Econometric Society, vol. 72(1), pages 159-217, January.
    27. Jessica Cohen & Pascaline Dupas, 2010. "Free Distribution or Cost-Sharing? Evidence from a Randomized Malaria Prevention Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 125(1), pages 1-45.
    28. Anup Malani, 2006. "Identifying Placebo Effects with Data from Clinical Trials," Journal of Political Economy, University of Chicago Press, vol. 114(2), pages 236-256, April.
    29. Malani, Anup, 2008. "Patient enrollment in medical trials: Selection bias in a randomized experiment," Journal of Econometrics, Elsevier, vol. 144(2), pages 341-351, June.
    30. Abhijit V. Banerjee, 1992. "A Simple Model of Herd Behavior," The Quarterly Journal of Economics, Oxford University Press, vol. 107(3), pages 797-817.
    31. Tomas Philipson & Larry V. Hedges, 1998. "Subject Evaluation in Social Experiments," Econometrica, Econometric Society, vol. 66(2), pages 381-408, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rema Hanna & Esther Duflo & Michael Greenstone, 2016. "Up in Smoke: The Influence of Household Behavior on the Long-Run Impact of Improved Cooking Stoves," American Economic Journal: Economic Policy, American Economic Association, vol. 8(1), pages 80-114, February.
    2. Abhijit Banerjee & Rukmini Banerji & James Berry & Esther Duflo & Harini Kannan & Shobhini Mukerji & Marc Shotland & Michael Walton, 2017. "From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 73-102, Fall.
    3. Amanda E. Kowalski, 2018. "Extrapolation using Selection and Moral Hazard Heterogeneity from within the Oregon Health Insurance Experiment," Cowles Foundation Discussion Papers 2135, Cowles Foundation for Research in Economics, Yale University.
    4. Davis, Alexander L. & Krishnamurti, Tamar, 2013. "The problems and solutions of predicting participation in energy efficiency programs," Applied Energy, Elsevier, vol. 111(C), pages 277-287.
    5. Amanda Kowalski, 2016. "Doing more when you're running LATE: Applying marginal treatment effect methods to examine treatment effect heterogeneity in experiments," Artefactual Field Experiments 00560, The Field Experiments Website.
    6. Eva Vivalt, 2015. "Heterogeneous Treatment Effects in Impact Evaluation," American Economic Review, American Economic Association, vol. 105(5), pages 467-470, May.
    7. Hunt Allcott, 2012. "Site Selection Bias in Program Evaluation," NBER Working Papers 18373, National Bureau of Economic Research, Inc.
    8. Juan Ortner & Sylvain Chassang, 2014. "Making Collusion Hard: Asymmetric Information as a Counter-Corruption Measure," Working Papers 064-2014, Princeton University, Department of Economics, Econometric Research Program..
    9. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2017. "Generalization in the Tropics: Development policy, randomized controlled trials, and external validity," Ruhr Economic Papers 716, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    10. B. Kelsey Jack, 2013. "Private Information and the Allocation of Land Use Subsidies in Malawi," American Economic Journal: Applied Economics, American Economic Association, vol. 5(3), pages 113-135, July.
    11. Raymond P. Guiteras & B. Kelsey Jack, 2014. "Incentives, Selection and Productivity in Labor Markets: Evidence from Rural Malawi," NBER Working Papers 19825, National Bureau of Economic Research, Inc.
    12. Bryan, Gharad & Karlan, Dean & Zinman, Jonathan, 2012. "You Can Pick Your Friends, but You Need to Watch Them: Loan Screening and Enforcement in a Referrals Field Experiment," Working Papers 99, Yale University, Department of Economics.
    13. Fischer, Gregory & Berry, James & Guiteras, Raymond, 2012. "Eliciting and utilizing willingness to pay: evidence from field trials in Northern Ghana," LSE Research Online Documents on Economics 47913, London School of Economics and Political Science, LSE Library.
    14. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2016. "Policy evaluation, randomized controlled trials, and external validity—A systematic review," Economics Letters, Elsevier, vol. 147(C), pages 51-54.
    15. Gani Aldashev & Georg Kirchsteiger & Alexander Sebald, 2012. "Assignment procedure biases in randomized policy experiments," Carlo Alberto Notebooks 292, Collegio Carlo Alberto.
    16. repec:eee:deveco:v:131:y:2018:i:c:p:42-61 is not listed on IDEAS
    17. Anup Malani & Tomas J. Philipson, 2011. "Can Medical Progress be Sustained? Implications of the Link Between Development and Output Markets," NBER Working Papers 17011, National Bureau of Economic Research, Inc.
    18. Fitzsimons, Emla & Malde, Bansi & Mesnard, Alice & Vera-Hernández, Marcos, 2012. "Household Responses to Information on Child Nutrition: Experimental Evidence from Malawi," CEPR Discussion Papers 8915, C.E.P.R. Discussion Papers.
    19. Amanda E. Kowalski, 2016. "Doing More When You're Running LATE: Applying Marginal Treatment Effect Methods to Examine Treatment Effect Heterogeneity in Experiments for the Young and Privately Insured?," Cowles Foundation Discussion Papers 2045, Cowles Foundation for Research in Economics, Yale University.
    20. B. Kelsey Jack & Paulina Oliva & Christopher Severen & Elizabeth Walker & Samuel Bell, 2015. "Technology Adoption Under Uncertainty: Take-Up and Subsequent Investment in Zambia," NBER Working Papers 21414, National Bureau of Economic Research, Inc.
    21. Cristina Corduneanu-Huci & Michael T. Dorsch & Paul Maarek, 2017. "Learning to constrain: Political competition and randomized controlled trials in development," THEMA Working Papers 2017-24, THEMA (THéorie Economique, Modélisation et Applications), Université de Cergy-Pontoise.
    22. Abhijit Banerjee & Sylvain Chassang & Erik Snowberg, 2016. "Decision Theoretic Approaches to Experiment Design and External Validity," NBER Working Papers 22167, National Bureau of Economic Research, Inc.

    More about this item

    JEL classification:

    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • D82 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Asymmetric and Private Information; Mechanism Design
    • O12 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Microeconomic Analyses of Economic Development
    • O22 - Economic Development, Innovation, Technological Change, and Growth - - Development Planning and Policy - - - Project Analysis

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aea:aecrev:v:102:y:2012:i:4:p:1279-1309. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Michael P. Albert). General contact details of provider: http://edirc.repec.org/data/aeaaaea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.