IDEAS home Printed from https://ideas.repec.org/p/pri/rpdevs/august_25.pdf.html
   My bibliography  Save this paper

Understanding and Misunderstanding Randomized Controlled Trials

Author

Listed:
  • Angus Deaton

    (Princeton University)

  • Nancy Cartwright

    (Durham University and University of California San Diego)

Abstract

RCTs are valuable tools whose use is spreading in economics and in other social sciences. They are seen as desirable aids in scientific discovery and for generating evidence for policy. Yet some of the enthusiasm for RCTs appears to be based on misunderstandings: that randomization provides a fair test by equalizing everything but the treatment and so allows a precise estimate of the treatment alone; that randomization is required to solve selection problems; that lack of blinding does little to compromise inference; and that statistical inference in RCTs is straightforward, because it requires only the comparison of two means. None of these statements is true. RCTs do indeed require minimal assumptions and can operate with little prior knowledge, an advantage when persuading distrustful audiences, but a crucial disadvantage for cumulative scientific progress, where randomization adds noise and undermines precision. The lack of connection between RCTs and other scientific knowledge makes it hard to use them outside of the exact context in which they are conducted. Yet, once they are seen as part of a cumulative program, they can play a role in building general knowledge and useful predictions, provided they are combined with other methods, including conceptual and theoretical development, to discover not "what works," but why things work. Unless we are prepared to make assumptions, and to stand on what we know, making statements that will be incredible to some, all the credibility of RCTs is for naught.

Suggested Citation

  • Angus Deaton & Nancy Cartwright, 2016. "Understanding and Misunderstanding Randomized Controlled Trials," Working Papers august_25.pdf, Princeton University, Woodrow Wilson School of Public and International Affairs, Research Program in Development Studies..
  • Handle: RePEc:pri:rpdevs:august_25.pdf
    as

    Download full text from publisher

    File URL: http://www.princeton.edu/~deaton/downloads/Deaton_Cartwright_RCTs_with_ABSTRACT_August_25.pdf
    Download Restriction: no

    Other versions of this item:

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. repec:wsi:jicepx:v:08:y:2017:i:02:n:s1793993317500077 is not listed on IDEAS
    2. Rao, Vijayendra & Ananthpur, Kripa & Malik, Kabir, 2017. "The Anatomy of Failure: An Ethnography of a Randomized Trial to Deepen Democracy in Rural India," World Development, Elsevier, vol. 99(C), pages 481-497.
    3. repec:pal:palcom:v:4:y:2018:i:1:d:10.1057_s41599-018-0069-9 is not listed on IDEAS
    4. Cristina Corduneanu-Huci & Michael T. Dorsch & Paul Maarek, 2017. "Learning to constrain: Political competition and randomized controlled trials in development," THEMA Working Papers 2017-24, THEMA (THéorie Economique, Modélisation et Applications), Université de Cergy-Pontoise.
    5. repec:eee:jeborg:v:145:y:2018:i:c:p:335-351 is not listed on IDEAS
    6. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2017. "Generalization in the Tropics: Development policy, randomized controlled trials, and external validity," Ruhr Economic Papers 716, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    7. Achilleas Vassilopoulos & Andreas C. Drichoutis & Rodolfo M. Nayga, Jr, 2018. "Loss Aversion, Expectations and Anchoring in the BDM Mechanism," Working Papers 2018-1, Agricultural University of Athens, Department Of Agricultural Economics.
    8. repec:aea:jecper:v:31:y:2017:i:4:p:125-44 is not listed on IDEAS
    9. Darwin Ugarte Ontiveros & Gustavo Canavire-Bacarreza & Luis Castro Peñarrieta, 2017. "Outliers in semi-parametric Estimation of Treatment Effects," DOCUMENTOS DE TRABAJO CIEF 015810, UNIVERSIDAD EAFIT.
    10. Vicky Chemutai & Hubert Escaith, 2017. "Measuring World Trade Organization (WTO) Accession Commitments and their Economic Effects," Journal of International Commerce, Economics and Policy (JICEP), World Scientific Publishing Co. Pte. Ltd., vol. 8(02), pages 1-27, June.
    11. repec:aea:aecrev:v:107:y:2017:i:5:p:282-86 is not listed on IDEAS
    12. Moshe Justman, 2016. "Economic Research and Education Policy: Project STAR and Class Size Reduction," Melbourne Institute Working Paper Series wp2016n37, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.
    13. Drichoutis, Andreas C. & Nayga, Rodolfo, 2017. "Economic rationality under cognitive load," MPRA Paper 81111, University Library of Munich, Germany.
    14. Andor, Mark Andreas & Gerster, Andreas & Peters, Jörg & Schmidt, Christoph M., 2017. "Social norms and energy conservation beyond the US," Ruhr Economic Papers 714, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    15. repec:kap:reveho:v:16:y:2018:i:1:d:10.1007_s11150-016-9353-x is not listed on IDEAS
    16. Omar Al-Ubaydli & John A. List & Danielle LoRe & Dana Suskind, 2017. "Scaling for Economists: Lessons from the Non-Adherence Problem in the Medical Literature," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 125-144, Fall.
    17. repec:eee:ecolet:v:167:y:2018:i:c:p:131-135 is not listed on IDEAS
    18. Omar Al-Ubaydli & John A. List & Dana L. Suskind, 2017. "What Can We Learn from Experiments? Understanding the Threats to the Scalability of Experimental Results," American Economic Review, American Economic Association, vol. 107(5), pages 282-286, May.
    19. repec:eee:jcjust:v:53:y:2017:i:c:p:110-116 is not listed on IDEAS

    More about this item

    JEL classification:

    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pri:rpdevs:august_25.pdf. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Bobray Bordelon). General contact details of provider: http://edirc.repec.org/data/rpprius.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.