IDEAS home Printed from https://ideas.repec.org/a/taf/jdevst/v47y2011i11p1619-1635.html
   My bibliography  Save this article

Against Excessive Rhetoric in Impact Assessment: Overstating the Case for Randomised Controlled Experiments

Author

Listed:
  • Paul Shaffer

Abstract

The recent attention afforded to randomisation, or Randomised Control Trials (RCTs), in impact assessment is a welcome development. The case for RCTs in international development, however, has been quite overstated. This article critically examines the seminal model underlying RCTs, the Holland-Rubin Framework, with a view to make four claims about RCTs: (i) they have limitations as conceptions of causation; (ii) their ‘idealised’ model of causal inference is undermined by implementation issues; (iii) they are not necessary to make internally valid statements about impact; and (iv) in general, they do not provide sufficient information for many purposes of impact assessment. The key argument is that ultimately, the choice of approach to impact assessment should be driven by the research question at hand and not by the alleged superiority of method.

Suggested Citation

  • Paul Shaffer, 2011. "Against Excessive Rhetoric in Impact Assessment: Overstating the Case for Randomised Controlled Experiments," Journal of Development Studies, Taylor & Francis Journals, vol. 47(11), pages 1619-1635.
    • Handle: RePEc:taf:jdevst:v:47:y:2011:i:11:p:1619-1635
    DOI: 10.1080/00220388.2010.514331
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/00220388.2010.514331
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1080/00220388.2010.514331?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2019. "All that Glitters is not Gold. The Political Economy of Randomized Evaluations in Development," Development and Change, International Institute of Social Studies, vol. 50(3), pages 735-762, May.
    2. Florent BEDECARRATS & Isabelle GUERIN & François ROUBAUD, 2017. "L'étalon-or des évaluations randomisées : économie politique des expérimentations aléatoires dans le domaine du développement," Working Paper 753120cd-506f-4c5f-80ed-7, Agence française de développement.
    3. Kabeer, Naila, 2020. "Women’s empowerment and economic development: a feminist critique of story telling practices in ‘Randomista' economics," LSE Research Online Documents on Economics 103880, London School of Economics and Political Science, LSE Library.
    4. Akram-Lodhi, A. Haroon, 2020. "“Follow the yellow brick road”?: Structural shortcomings in randomized control trials," World Development, Elsevier, vol. 127(C).
    5. Kabeer, Naila, 2020. "‘Misbehaving’ RCTs: The confounding problem of human agency," World Development, Elsevier, vol. 127(C).
    6. Kabeer, Naila, 2020. "Misbehaving’ RCTs: the confounding problem of human agency," LSE Research Online Documents on Economics 102940, London School of Economics and Political Science, LSE Library.
    7. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2015. "The gold standard for randomized evaluations: from discussion of method to political economy," Working Papers DT/2015/01, DIAL (Développement, Institutions et Mondialisation).
    8. Lota Tamini & Ibrahima Bocoum & Ghislain Auger & Kotchikpa Gabriel Lawin & Arahama Traoré, 2019. "Enhanced Microfinance Services and Agricultural Best Management Practices: What Benefits for Smallholders Farmers? An Evidence from Burkina Faso," CIRANO Working Papers 2019s-11, CIRANO.
    9. Woolcock, Michael, 2013. "Using Case Studies to Explore the External Validity of 'Complex' Development Interventions," Working Paper Series rwp13-048, Harvard University, John F. Kennedy School of Government.
    10. Paul Shaffer, 2015. "Two Concepts of Causation: Implications for Poverty," Development and Change, International Institute of Social Studies, vol. 46(1), pages 148-166, January.
    11. Florent Bedecarrats & Isabelle Guérin & François Roubaud, 2017. "L'étalon-or des évaluations randomisées : du discours de la méthode à l'économie politique," Working Papers ird-01445209, HAL.
    12. McHugh, Neil & Biosca, Olga & Donaldson, Cam, 2015. "Microfinance, health and randomised trials," Health Economics Working Paper Series 201501, Glasgow Caledonian University, Yunus Centre.
    13. Kabeer, Naila, 2020. "Women's Empowerment and Economic Development: A Feminist Critique of Storytelling Practices in "Randomista" Economics: a feminist critique of storytelling practices in “randomista” economics," LSE Research Online Documents on Economics 104600, London School of Economics and Political Science, LSE Library.
    14. Woolcock, Michael, 2013. "Using Case Studies to Explore the External Validity of 'Complex' Development Interventions," Working Paper Series rwp13-048, Harvard University, John F. Kennedy School of Government.
    15. Colin Kirkpatrick, 2012. "Economic Governance: Improving the Economic and Regulatory Environment for Supporting Private Sector Activity," WIDER Working Paper Series wp-2012-108, World Institute for Development Economic Research (UNU-WIDER).
    16. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2015. "The gold standard for randomised evaluations: from discussion of method to political economics," Working Papers CEB 15-009, ULB -- Universite Libre de Bruxelles.
    17. Willan, Samantha & Gibbs, Andrew & Shai, Nwabisa & Ntini, Nolwazi & Petersen, Inge & Jewkes, Rachel, 2020. "Did young women in South African informal settlements display increased agency after participating in the Stepping Stones and Creating Futures intervention? A qualitative evaluation," Social Science & Medicine, Elsevier, vol. 265(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:jdevst:v:47:y:2011:i:11:p:1619-1635. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/FJDS20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.