IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/27554.html
   My bibliography  Save this paper

Should the Randomistas (Continue to) Rule?

Author

Listed:
  • Martin Ravallion

Abstract

The rising popularity of randomized controlled trials (RCTs) in development applications has come with continuing debates about the merits of this approach. The paper takes stock of the issues. It argues that an unconditional preference for RCTs is questionable on three main counts. First, the case for such a preference is unclear on a priori grounds. For example, with a given budget, even a biased observational study can come closer to the truth than a costly RCT. Second, the ethical objections to RCTs have not been properly addressed by advocates. Third, there is a risk of distorting the evidence-base for informing policymaking, given that an insistence on RCTs generates selection bias in what gets evaluated. Going forward, pressing knowledge gaps should drive the questions asked and how they are answered, not the methodological preferences of some researchers. The gold standard is the best method for the question at hand.

Suggested Citation

  • Martin Ravallion, 2020. "Should the Randomistas (Continue to) Rule?," NBER Working Papers 27554, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:27554
    Note: DEV
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w27554.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Stéphane J. Baele, 2013. "The ethics of New Development Economics: is the Experimental Approach to Development Economics morally wrong?," The Journal of Philosophical Economics, Bucharest Academy of Economic Studies, The Journal of Philosophical Economics, vol. 7(1), November.
    2. Abhijit V. Banerjee & Esther Duflo, 2009. "The Experimental Approach to Development Economics," Annual Review of Economics, Annual Reviews, vol. 1(1), pages 151-178, May.
    3. Arthur Alik‐Lagrange & Martin Ravallion, 2019. "Estimating within‐cluster spillover effects using a cluster randomization with application to knowledge diffusion in rural India," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 34(1), pages 110-128, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Fatoumata Nankoto Cissé, 2022. "How impact evaluation methods influence the outcomes of development projects? Evidence from a meta-analysis on decentralized solar nano projects," Post-Print halshs-03623394, HAL.
    2. Brodeur, Abel & Cook, Nikolai M. & Hartley, Jonathan S. & Heyes, Anthony, 2022. "Do Pre-Registration and Pre-analysis Plans Reduce p-Hacking and Publication Bias?," GLO Discussion Paper Series 1147, Global Labor Organization (GLO).
    3. Charles Yuji Horioka, 2021. "Is the selfish life-cycle model more applicable in Japan and, if so, why? A literature survey," Review of Economics of the Household, Springer, vol. 19(1), pages 157-187, March.
    4. Giovanni Abbiati & Davide Azzolini & Anja Balanskat & Katja Engelhart & Daniela Piazzalunga & Enrico Rettore & Patricia Wastiau, 2023. "Effects of an Online Self-Assessment Tool on Teachers’ Digital Competencies," FBK-IRVAPP Working Papers 2023-01, Research Institute for the Evaluation of Public Policies (IRVAPP), Bruno Kessler Foundation.
    5. Juan Andrés Cabral & Florencia Iara Pucci, 2020. "¿Cuál es el alcance de la revolución de la credibilidad?," Asociación Argentina de Economía Política: Working Papers 4318, Asociación Argentina de Economía Política.
    6. Fatoumata Nankoto Cissé, 2022. "How impact evaluation methods influence the outcomes of development projects? Evidence from a meta-analysis on decentralized solar nano projects," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-03623394, HAL.
    7. Carmen Camacho & Hannes Tepper, 2023. "Do this or do that? A model to prioritize reforms," Working Papers halshs-04005785, HAL.
    8. Felix Naschold & Christopher B. Barrett, 2020. "A stochastic dominance approach to program evaluation with an application to child nutritional status in Kenya," Agricultural Economics, International Association of Agricultural Economists, vol. 51(6), pages 871-886, November.
    9. Ravallion, Martin, 2020. "Highly prized experiments," World Development, Elsevier, vol. 127(C).
    10. Vikram Tyagi & Sophie Webber, 2021. "A rusting gold standard: Failures in an Indonesian RCT, and the implications for poverty reduction," Environment and Planning A, , vol. 53(5), pages 992-1011, August.
    11. Clotilde Mahé & Philipp Hessel, 2022. "School-age exposure to conditional cash transfers and adult mental health: Evidence from Mexico’s Progresa," Documentos de trabajo 20155, Escuela de Gobierno - Universidad de los Andes.
    12. Hanushek, Eric A., 2021. "Addressing cross-national generalizability in educational impact evaluation," International Journal of Educational Development, Elsevier, vol. 80(C).
    13. Brodeur, Abel & Cook, Nikolai M. & Hartley, Jonathan S. & Heyes, Anthony, 2023. "Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias?: Evidence from 15,992 Test Statistics and Suggestions for Improvement," GLO Discussion Paper Series 1147 [pre.], Global Labor Organization (GLO).
    14. J. Nicolas Hernandez-Aguilera & Max Mauerman & Alexandra Herrera & Kathryn Vasilaky & Walter Baethgen & Ana Maria Loboguerrero & Rahel Diro & Yohana Tesfamariam Tekeste & Daniel Osgood, 2020. "Games and Fieldwork in Agriculture: A Systematic Review of the 21st Century in Economics and Social Science," Games, MDPI, vol. 11(4), pages 1-22, October.
    15. Simon Feeny & Gill Westhorp & Emma Williams, 2023. "Understanding sustainable outcomes in international development: Towards a realist evaluation framework," Journal of International Development, John Wiley & Sons, Ltd., vol. 35(1), pages 21-42, January.
    16. Carmen Camacho & Hannes Tepper, 2023. "Do this or do that? A model to prioritize reforms," PSE Working Papers halshs-04005785, HAL.
    17. Fatoumata Nankoto Cissé, 2022. "How impact evaluation methods influence the outcomes of development projects? Evidence from a meta-analysis on decentralized solar nano projects," Documents de travail du Centre d'Economie de la Sorbonne 22008, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
    18. Fuchs, Lisa Elena & Orero, Levi & Apondi, Victoria Atieno & Kipkorir, Lang'at, 2021. "How to stop wasting money in international development: Using a structured group selection approach to counter procedural inefficiency," World Development Perspectives, Elsevier, vol. 24(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michel Abramowicz & Ariane Szafarz, 2019. "Ethics of Randomized Controlled Trials: Should Economists Care about Equipoise?," Working Papers CEB 19-017, ULB -- Universite Libre de Bruxelles.
    2. Martin Ravallion, 2022. "On the Gains from Tradable Benefits‐in‐kind: Evidence for Workfare in India," Economica, London School of Economics and Political Science, vol. 89(355), pages 770-787, July.
    3. Ashish Arora & Michelle Gittelman & Sarah Kaplan & John Lynch & Will Mitchell & Nicolaj Siggelkow & Aaron K. Chatterji & Michael Findley & Nathan M. Jensen & Stephan Meier & Daniel Nielson, 2016. "Field experiments in strategy research," Strategic Management Journal, Wiley Blackwell, vol. 37(1), pages 116-132, January.
    4. James J. Heckman, 1991. "Randomization and Social Policy Evaluation Revisited," NBER Technical Working Papers 0107, National Bureau of Economic Research, Inc.
    5. Pedro Carneiro & Sokbae Lee & Daniel Wilhelm, 2020. "Optimal data collection for randomized control trials [Microcredit impacts: Evidence from a randomized microcredit program placement experiment by Compartamos Banco]," The Econometrics Journal, Royal Economic Society, vol. 23(1), pages 1-31.
    6. Su, Huei-Chun & Colander, David, 2021. "The Economist As Scientist, Engineer, Or Plumber?," Journal of the History of Economic Thought, Cambridge University Press, vol. 43(2), pages 297-312, June.
    7. Guizar-Mateos, Isai & Miranda, Mario J. & Gonzalez-Vega, Claudio, 2013. "The Role of Credit and Deposits in the Dynamics of Technology Decisions and Poverty Traps," 2013 Annual Meeting, August 4-6, 2013, Washington, D.C. 149860, Agricultural and Applied Economics Association.
    8. Stefano DellaVigna & Elizabeth Linos, 2022. "RCTs to Scale: Comprehensive Evidence From Two Nudge Units," Econometrica, Econometric Society, vol. 90(1), pages 81-116, January.
    9. Julie Le Gallo & Yannick L'Horty & Pascale Petit, 2014. "Does subsidising young people to learn to drive promote social inclusion? Evidence from a large controlled experiment in France," TEPP Working Paper 2014-15, TEPP.
    10. James J. Heckman, 1991. "Randomization and Social Policy Evaluation Revisited," NBER Technical Working Papers 0107, National Bureau of Economic Research, Inc.
    11. Nguyen Viet, Cuong, 2006. "An Introduction to Alternative Methods in Program Impact Evaluation," MPRA Paper 24900, University Library of Munich, Germany.
    12. Mariana Blanco & Juan F. Vargas, 2010. "Empowering IDPs with SMS: A Randomized Controlled Trial in Bogota," Research Department Publications 4704, Inter-American Development Bank, Research Department.
    13. Anna M. Wilke & Donald P. Green & Jasper Cooper, 2020. "A placebo design to detect spillovers from an education–entertainment experiment in Uganda," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(3), pages 1075-1096, June.
    14. Cornwall, Andrea & Aghajanian, Alia, 2017. "How to Find out What’s Really Going On: Understanding Impact through Participatory Process Evaluation," World Development, Elsevier, vol. 99(C), pages 173-185.
    15. Steve Gibbons & Max Nathan & Henry G. Overman, 2014. "Evaluating Spatial Policies," SERC Policy Papers 012, Centre for Economic Performance, LSE.
    16. Paolo Pinotti, 0. "The Credibility Revolution in the Empirical Analysis of Crime," Italian Economic Journal: A Continuation of Rivista Italiana degli Economisti and Giornale degli Economisti, Springer;Società Italiana degli Economisti (Italian Economic Association), vol. 0, pages 1-14.
    17. Margherita Scarlato, 2010. "New welfare policy instruments: international experiences and implications for social enterprises," Euricse Working Papers 10013, Euricse (European Research Institute on Cooperative and Social Enterprises).
    18. Izzo, Federica & Dewan, Torun & Wolton, Stephane, 2022. "Cumulative knowledge in the social sciences: The case of improving voters' information," MPRA Paper 112559, University Library of Munich, Germany.
    19. Sana Ullah & Muhammad Tariq Majeed & Babur Wasim Arif, 2021. "Social capital and firms’ choice of financing under credit constraints: microeconomic evidence from Pakistan," DECISION: Official Journal of the Indian Institute of Management Calcutta, Springer;Indian Institute of Management Calcutta, vol. 48(1), pages 3-13, March.
    20. Martin Prowse, 2010. "Integrating reflexivity into livelihoods research," Progress in Development Studies, , vol. 10(3), pages 211-231, July.

    More about this item

    JEL classification:

    • B41 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Economic Methodology
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • O22 - Economic Development, Innovation, Technological Change, and Growth - - Development Planning and Policy - - - Project Analysis

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:27554. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.