IDEAS home Printed from https://ideas.repec.org/p/hal/journl/ird-02112849.html
   My bibliography  Save this paper

All that glitters is not gold : the political economy of randomized evaluations in development

Author

Listed:
  • Florent Bédécarrats

    (AFD - Agence française de développement)

  • Isabelle Guérin

    () (CESSMA UMRD 245 - Centre d'études en sciences sociales sur les mondes africains, américains et asiatiques - IRD - Institut de Recherche pour le Développement - Inalco - Institut National des Langues et Civilisations Orientales - UPD7 - Université Paris Diderot - Paris 7)

  • François Roubaud

    (LEDa - Laboratoire d'Economie de Dauphine - Université Paris Dauphine-PSL)

Abstract

Randomized control trials (RCTs) have a narrow scope, restricted to basic intervention schemes. Experimental designs also display specific biases and political uses when implemented in the real world. Despite these limitations, the method has been advertised as the gold standard to evaluate development policies. This article adopts a political economy approach to explore this paradox. It argues that the success of RCTs is driven mainly by a new scientific business model based on a mix of simplicity and mathematical rigour, media and donor appeal, and academic and financial returns. This in turn meets current interests and preferences in the academic world and the donor community.

Suggested Citation

  • Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2019. "All that glitters is not gold : the political economy of randomized evaluations in development," Post-Print ird-02112849, HAL.
  • Handle: RePEc:hal:journl:ird-02112849
    DOI: 10.1111/dech.12378
    Note: View the original document on HAL open archive server: https://hal.ird.fr/ird-02112849
    as

    Download full text from publisher

    File URL: https://hal.ird.fr/ird-02112849/document
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. ., 2017. "The development of an applied economist," Chapters, in: The Value of Applied Economics, chapter 2, pages 15-32, Edward Elgar Publishing.
    2. Daron Acemoglu, 2010. "Theory, General Equilibrium, and Political Economy in Development Economics," Journal of Economic Perspectives, American Economic Association, vol. 24(3), pages 17-32, Summer.
    3. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    4. Woolcock, Michael, 2013. "Using Case Studies to Explore the External Validity of 'Complex' Development Interventions," Working Paper Series rwp13-048, Harvard University, John F. Kennedy School of Government.
    5. James J. Heckman, 1991. "Randomization and Social Policy Evaluation Revisited," NBER Technical Working Papers 0107, National Bureau of Economic Research, Inc.
    6. John A. List & Robert Metcalfe, 2014. "Field experiments in the developed world: an introduction," Oxford Review of Economic Policy, Oxford University Press, vol. 30(4), pages 585-596.
    7. James J. Heckman & Edward Vytlacil, 2005. "Structural Equations, Treatment Effects, and Econometric Policy Evaluation," Econometrica, Econometric Society, vol. 73(3), pages 669-738, May.
    8. Solène Morvant-Roux & Isabelle Guérin & Marc Roesch & Jean-Yves Moisseron, 2014. "Adding value to randomization with qualitative analysis : the case of microcredit in rural Morocco," Post-Print ird-01471911, HAL.
    9. Lant Pritchett & Salimah Samji & Jeffrey Hammer, 2012. "It’s All About MeE: Using Structured Experiential Learning (‘e’) to Crawl the Design Space," CID Working Papers 249, Center for International Development at Harvard University.
    10. Peter Boone & Alex Eble & Diana Elbourne, 2013. "Risk and Evidence of Bias in Randomized Controlled Trials in Economics," CEP Discussion Papers dp1240, Centre for Economic Performance, LSE.
    11. Duflo, Esther & Dupas, Pascaline & Kremer, Michael, 2015. "School governance, teacher incentives, and pupil–teacher ratios: Experimental evidence from Kenyan primary schools," Journal of Public Economics, Elsevier, vol. 123(C), pages 92-110.
    12. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2015. "The gold standard for randomised evaluations: from discussion of method to political economics," Working Papers CEB 15-009, ULB -- Universite Libre de Bruxelles.
    13. Edward Miguel & Michael Kremer, 2004. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities," Econometrica, Econometric Society, vol. 72(1), pages 159-217, January.
    14. Abhijit Banerjee & Esther Duflo & Rachel Glennerster & Cynthia Kinnan, 2015. "The Miracle of Microfinance? Evidence from a Randomized Evaluation," American Economic Journal: Applied Economics, American Economic Association, vol. 7(1), pages 22-53, January.
    15. Karthik Muralidharan & Venkatesh Sundararaman, 2011. "Teacher Performance Pay: Experimental Evidence from India," Journal of Political Economy, University of Chicago Press, vol. 119(1), pages 39-77.
    16. Glenn W. Harrison, 2011. "Randomisation and Its Discontents," Journal of African Economies, Centre for the Study of African Economies (CSAE), vol. 20(4), pages 626-652, August.
    17. Tanguy Bernard & Jocelyne Delarue & Jean-David Naudet, 2012. "Impact evaluations: a tool for accountability? Lessons from experience at Agence Française de Développement," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 4(2), pages 314-327, June.
    18. Michael Rosholm & Lars Skipper, 2009. "Is labour market training a curse for the unemployed? Evidence from a social experiment," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 24(2), pages 338-365, March.
    19. Paul Shaffer, 2011. "Against Excessive Rhetoric in Impact Assessment: Overstating the Case for Randomised Controlled Experiments," Journal of Development Studies, Taylor & Francis Journals, vol. 47(11), pages 1619-1635.
    20. Christopher B. Barrett & Michael R. Carter, 2010. "The Power and Pitfalls of Experiments in Development Economics: Some Non-random Reflections," Applied Economic Perspectives and Policy, Agricultural and Applied Economics Association, vol. 32(4), pages 515-548.
    21. Easterly, William, 2007. "The White Man's Burden: Why the West's Efforts to Aid the Rest Have Done So Much Ill And So Little Good," OUP Catalogue, Oxford University Press, number 9780199226115.
    22. Deaton, Angus & Cartwright, Nancy, 2018. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
    23. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2015. "The gold standard for randomized evaluations: from discussion of method to political economy," Working Papers DT/2015/01, DIAL (Développement, Institutions et Mondialisation).
    24. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    25. Abhijit Banerjee & Dean Karlan & Jonathan Zinman, 2015. "Six Randomized Evaluations of Microcredit: Introduction and Further Steps," American Economic Journal: Applied Economics, American Economic Association, vol. 7(1), pages 1-21, January.
    26. Lant Pritchett, Justin Sandefur, 2013. "Context Matters for Size: Why External Validity Claims and Development Practice Don't Mix-Working Paper 336," Working Papers 336, Center for Global Development.
    27. Rachel Glennerster, 2012. "The Power of Evidence: Improving the Effectiveness of Government by Investing in More Rigorous Evaluation," National Institute Economic Review, National Institute of Economic and Social Research, vol. 219(1), pages 4-14, January.
    28. Morvant-Roux, Solène & Guérin, Isabelle & Roesch, Marc & Moisseron, Jean-Yves, 2014. "Adding Value to Randomization with Qualitative Analysis: The Case of Microcredit in Rural Morocco," World Development, Elsevier, vol. 56(C), pages 302-312.
    29. Levine, David & Polimeni, Rachel & Ramage, Ian, 2016. "Insuring health or insuring wealth? An experimental evaluation of health insurance in rural Cambodia," Journal of Development Economics, Elsevier, vol. 119(C), pages 1-15.
    30. Lant Pritchett & Justin Sandefur, 2015. "Learning from Experiments When Context Matters," American Economic Review, American Economic Association, vol. 105(5), pages 471-475, May.
    31. Martin Ravallion, 2009. "Evaluation in the Practice of Development," World Bank Research Observer, World Bank Group, vol. 24(1), pages 29-53, March.
    32. Labrousse, Agnès, 2010. "Nouvelle économie du développement et essais cliniques randomisés : une mise en perspective d’un outil de preuve et de gouvernement," Revue de la Régulation - Capitalisme, institutions, pouvoirs, Association Recherche et Régulation, vol. 7.
    33. James Heckman & Jeffrey Smith & Christopher Taber, 1998. "Accounting For Dropouts In Evaluations Of Social Programs," The Review of Economics and Statistics, MIT Press, vol. 80(1), pages 1-14, February.
    34. Angus Deaton & Nancy Cartwright, 2016. "Understanding and Misunderstanding Randomized Controlled Trials," Working Papers august_25.pdf, Princeton University, Woodrow Wilson School of Public and International Affairs, Research Program in Development Studies..
    35. Rodrik, Dani, 2008. "The New Development Economics: We Shall Experiment, but How Shall We Learn?," Working Paper Series rwp08-055, Harvard University, John F. Kennedy School of Government.
    Full references (including those not matched with items on IDEAS)

    More about this item

    Keywords

    Experimental method; Political economy; Development JEL codes: A11; B41; C18; C93; D72; O10; Randomized control trials; methodology; epistemology; Impact evaluation; Randomised control trials;

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • B41 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Economic Methodology
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • D72 - Microeconomics - - Analysis of Collective Decision-Making - - - Political Processes: Rent-seeking, Lobbying, Elections, Legislatures, and Voting Behavior
    • O10 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:journl:ird-02112849. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (CCSD). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.