IDEAS home Printed from https://ideas.repec.org/a/oup/apecpp/v32y2010i4p515-548.html
   My bibliography  Save this article

The Power and Pitfalls of Experiments in Development Economics: Some Non-random Reflections

Author

Listed:
  • Christopher B. Barrett
  • Michael R. Carter

Abstract

Impact evaluation based on randomized controlled trials (RCTs) offers a powerful tool that has fundamentally reshaped development economics by offering novel solutions to long-standing problems of weak causal identification. Nonetheless, RCTs suffer important and underappreciated pitfalls, some of which are intrinsic to the method when applied to economic problems, others that are the result of methodological boosterism. Among the pitfalls are ethical dilemmas, uncontrollable treatments that result in a 'faux exogeneity,' distortion of the research agenda, and a tendency to estimate interventions' abstract efficacy rather than their effectiveness in practice. We illustrate these points through the literature on smallholder capital access and productivity growth. Ultimately, we argue for a methodological pluralism that recognizes all identification strategies' limitations. Copyright 2010, Oxford University Press.

Suggested Citation

  • Christopher B. Barrett & Michael R. Carter, 2010. "The Power and Pitfalls of Experiments in Development Economics: Some Non-random Reflections," Applied Economic Perspectives and Policy, Agricultural and Applied Economics Association, vol. 32(4), pages 515-548.
  • Handle: RePEc:oup:apecpp:v:32:y:2010:i:4:p:515-548
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/aepp/ppq023
    Download Restriction: Access to full text is restricted to subscribers.

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. repec:oup:revage:v:26:y:2004:i:4:p:539-551. is not listed on IDEAS
    2. Bruce L. Gardner, 1976. "Futures Prices in Supply Analysis," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 58(1), pages 81-84.
    3. Hongli Feng & Bruce A. Babcock, 2010. "Impacts of Ethanol on Planted Acreage in Market Equilibrium," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 92(3), pages 789-802.
    4. Torre Ugarte, Daniel de la & Sanford, Scott & Skinner, Robert A. & Westcott, Paul C. & Lin, William W., 2000. "Supply Response Under The 1996 Farm Act And Implications For The U.S. Field Crops Sector," Technical Bulletins 33568, United States Department of Agriculture, Economic Research Service.
    5. Tweeten, Luther G & Quance, C Leroy, 1969. "Positivistic Measures of Aggregate Supply Elasticities: Some New Approaches," American Economic Review, American Economic Association, vol. 59(2), pages 175-183, May.
    6. Davison, Cecil W. & Crowder, Bradley M., 1991. "Northeast Soybean Acreage Response Using Expected Net Returns," Northeastern Journal of Agricultural and Resource Economics, Northeastern Agricultural and Resource Economics Association, vol. 20(1), April.
    7. David R. Lee & Peter G. Helmberger, 1985. "Estimating Supply Response in the Presence of Farm Programs," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 67(2), pages 193-203.
    8. Ahmed, Syud Amer & Thomas Hertel & Ruben Lubowski, 2009. "Calibration of a Land Cover Supply Function Using Transition Probabilities," GTAP Research Memoranda 2947, Center for Global Trade Analysis, Department of Agricultural Economics, Purdue University.
    9. Luther G. Tweeten & C. Leroy Quance, 1969. "Positivistic Measures of Aggregate Supply Elasticities: Some New Approaches," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 51(2), pages 342-352.
    10. Tristan Brown & Amani Elobeid & Jerome Dumortier & Dermot J. Hayes, 2010. "Market Impact of Domestic Offset Programs," Food and Agricultural Policy Research Institute (FAPRI) Publications 10-wp502, Food and Agricultural Policy Research Institute (FAPRI) at Iowa State University.
    11. Chad E. Hart & Bruce A. Babcock, 2005. "Loan Deficiency Payments versus Countercyclical Payments: Do We Need Both for a Price Safety Net?," Center for Agricultural and Rural Development (CARD) Publications 05-bp44, Center for Agricultural and Rural Development (CARD) at Iowa State University.
    12. J. P. Houck & M. E. Ryan, 1972. "Supply Analysis for Corn in the United States: The Impact of Changing Government Programs," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 54(2), pages 184-191.
    13. Kevin C. Dhuyvetter, 2004. "Using Formula Prices in the Absence of Publicly Reported Prices: An Application for Segregated Early Weaned Pigs," Review of Agricultural Economics, Agricultural and Applied Economics Association, vol. 26(4), pages 539-551.
    14. Houck, James P. & Ryan, Mary E., 1972. "Supply Analysis For Corn In The United States: The Impact Of Changing Government Programs," Staff Papers 13554, University of Minnesota, Department of Applied Economics.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. repec:bla:agecon:v:48:y:2017:i:3:p:317-326 is not listed on IDEAS
    2. Barrett, Christopher B. & Bachke, Maren E. & Bellemare, Marc F. & Michelson, Hope C. & Narayanan, Sudha & Walker, Thomas F., 2010. "Smallholder Participation in Agricultural Value Chains: Comparative Evidence from Three Continents," MPRA Paper 27829, University Library of Munich, Germany.
    3. Garg, Teevrat & Barrett, Christopher B. & Gómez, Miguel I. & Lentz, Erin C. & Violette, William J., 2013. "Market Prices and Food Aid Local and Regional Procurement and Distribution: A Multi-Country Analysis," World Development, Elsevier, vol. 49(C), pages 19-29.
    4. Sarah Baird & Craig McIntosh & Berk Özler, 2011. "Cash or Condition? Evidence from a Cash Transfer Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 126(4), pages 1709-1753.
    5. Florent Bedecarrats & Isabelle Guérin & François Roubaud, 2017. "L'étalon-or des évaluations randomisées : du discours de la méthode à l'économie politique," Working Papers ird-01445209, HAL.
    6. Agnes Quisumbing & Bob Baulch & Neha Kumar, 2011. "Evaluating the long-term impact of anti-poverty interventions in Bangladesh: an overview," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 3(2), pages 153-174.
    7. Cadot, Olivier & Fernandes, Ana & Gourdon, Julien & Mattoo, Aaditya, 2011. "Impact Evaluation of Trade Interventions: Paving the Way," CEPR Discussion Papers 8638, C.E.P.R. Discussion Papers.
    8. Bamberger, Michael & Tarsilla, Michele & Hesse-Biber, Sharlene, 2016. "Why so many “rigorous” evaluations fail to identify unintended consequences of development programs: How mixed methods can contribute," Evaluation and Program Planning, Elsevier, vol. 55(C), pages 155-162.
    9. Stéphane J. Baele, 2013. "The ethics of New Development Economics: is the Experimental Approach to Development Economics morally wrong?," The Journal of Philosophical Economics, Bucharest Academy of Economic Studies, The Journal of Philosophical Economics, vol. 7(1), November.
    10. Holden, Stein T. & Westberg, Nina Bruvik, 2016. "Exploring technology use under climate risk and shocks through an experimental lens," African Journal of Agricultural and Resource Economics, African Association of Agricultural Economists, vol. 11(1), March.
    11. Marc F. Bellemare, 2013. "The Productivity Impacts of Formal and Informal Land Rights: Evidence from Madagascar," Land Economics, University of Wisconsin Press, vol. 89(2), pages 272-290.
    12. Barrientos Armando & Villa Juan Miguel, 2015. "Evaluating Antipoverty Transfer Programmes in Latin America and Sub-Saharan Africa. Better Policies? Better Politics?," Journal of Globalization and Development, De Gruyter, vol. 6(1), pages 147-179, June.
    13. Gentilini, Ugo & Omamo, Steven Were, 2011. "Social protection 2.0: Exploring issues, evidence and debates in a globalizing world," Food Policy, Elsevier, vol. 36(3), pages 329-340, June.
    14. González-Flores, Mario & Bravo-Ureta, Boris E. & Solís, Daniel & Winters, Paul, 2014. "The impact of high value markets on smallholder productivity in the Ecuadorean Sierra: A Stochastic Production Frontier approach correcting for selectivity bias," Food Policy, Elsevier, vol. 44(C), pages 237-247.
    15. Henrik Hansen & Ninja Ritter Klejnstrup & Ole Winckler Andersen, 2011. "A Comparison of Model-based and Design-based Impact Evaluations of Interventions in Developing Countries," IFRO Working Paper 2011/16, University of Copenhagen, Department of Food and Resource Economics.
    16. Beliyou Haile & Carlo Azzarri & Cleo Roberts & David J. Spielman, 2017. "Targeting, bias, and expected impact of complex innovations on developing-country agriculture: evidence from Malawi," Agricultural Economics, International Association of Agricultural Economists, vol. 48(3), pages 317-326, May.
    17. Bellon, Mauricio R. & Gotor, Elisabetta & Caracciolo, Francesco, 2015. "Assessing the Effectiveness of Projects Supporting On-Farm Conservation of Native Crops: Evidence From the High Andes of South America," World Development, Elsevier, vol. 70(C), pages 162-176.
    18. Jacopo, Bonan & Stefano, Pareglio & Valentina, Rotondi, 2015. "Extension Services, Production and Welfare: Evidence from a Field Experiment in Ethiopia," Working Papers 312, University of Milano-Bicocca, Department of Economics, revised 30 Oct 2015.
    19. Ricker-Gilbert, Jacob & Jones, Michael, 2015. "Does storage technology affect adoption of improved maize varieties in Africa? Insights from Malawi’s input subsidy program," Food Policy, Elsevier, vol. 50(C), pages 92-105.
    20. Dietrich, Stephan & Malerba, Daniele & Barrientos, Armando & Gassmann, Franziska, 2017. "Rates of return to antipoverty transfers in Uganda," MERIT Working Papers 040, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    21. Van den Broeck, Goedele & Maertens, Miet, 2015. "Female Employment reduces Fertility in Rural Senegal," 2015 Conference, August 9-14, 2015, Milan, Italy 212206, International Association of Agricultural Economists.
    22. Smith, Lisa C. & Khan, Faheem & Frankenberger, Timothy R. & Wadud, A.K.M. Abdul, 2013. "Admissible Evidence in the Court of Development Evaluation? The Impact of CARE’s SHOUHARDO Project on Child Stunting in Bangladesh," World Development, Elsevier, vol. 41(C), pages 196-216.
    23. Jimena Hurtado, 2014. "Albert O. Hirschman y la economía del desarrollo: lecciones para el presente," REVISTA CUADERNOS DE ECONOMÍA, UN - RCE - CID, June.
    24. Barrett, Christopher B. & Bachke, Maren E. & Bellemare, Marc F. & Michelson, Hope C. & Narayanan, Sudha & Walker, Thomas F., 2012. "Smallholder Participation in Contract Farming: Comparative Evidence from Five Countries," World Development, Elsevier, vol. 40(4), pages 715-730.
    25. Boris E. Bravo-Ureta, 2014. "Stochastic frontiers, productivity effects and development projects," Economics and Business Letters, Oviedo University Press, vol. 3(1), pages 51-58.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:apecpp:v:32:y:2010:i:4:p:515-548. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Oxford University Press) or (Christopher F. Baum). General contact details of provider: http://edirc.repec.org/data/aaeaaea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.