IDEAS home Printed from
MyIDEAS: Login to save this paper or follow this series

In pursuit of balance : randomization in practice in development field experiments

  • Bruhn, Miriam
  • McKenzie, David

Randomized experiments are increasingly used in development economics, with researchers now facing the question of not just whether to randomize, but how to do so. Pure random assignment guarantees that the treatment and control groups will have identical characteristics on average, but in any particular random allocation, the two groups will differ along some dimensions. Methods used to pursue greater balance include stratification, pair-wise matching, and re-randomization. This paper presents new evidence on the randomization methods used in existing randomized experiments, and carries out simulations in order to provide guidance for researchers. Three main results emerge. First, many researchers are not controlling for the method of randomization in their analysis. The authors show this leads to tests with incorrect size, and can result in lower power than if a pure random draw was used. Second, they find that in samples of 300 or more, the different randomization methods perform similarly in terms of achieving balance on many future outcomes of interest. However, for very persistent outcome variables and in smaller sample sizes, pair-wise matching and stratification perform best. Third, the analysis suggests that on balance the re-randomization methods common in practice are less desirable than other methods, such as matching.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL:
Download Restriction: no

Paper provided by The World Bank in its series Policy Research Working Paper Series with number 4752.

in new window

Date of creation: 01 Oct 2008
Date of revision:
Handle: RePEc:wbk:wbrwps:4752
Contact details of provider: Postal: 1818 H Street, N.W., Washington, DC 20433
Phone: (202) 477-1234
Web page:

More information through EDIRC

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. repec:oup:qjecon:v:124:y:2009:i:2:p:735-769 is not listed on IDEAS
  2. Skoufias, Emmanuel, 2005. "PROGRESA and its impacts on the welfare of rural households in Mexico:," Research reports 139, International Food Policy Research Institute (IFPRI).
  3. Steven D. Levitt & John A. List, 2008. "Field Experiments in Economics: The Past, The Present, and The Future," NBER Working Papers 14356, National Bureau of Economic Research, Inc.
  4. Kosuke Imai & Gary King & Elizabeth A. Stuart, 2008. "Misunderstandings between experimentalists and observationalists about causal inference," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 171(2), pages 481-502.
  5. Gary Burtless, 1995. "The Case for Randomized Field Trials in Economic and Policy Research," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 63-84, Spring.
  6. Eric Zitzewitz & Michael Kremer & Paul Glewwe & Sylvie Moulin, 2004. "Retrospective vs. prospective analyses of school inputs: The case of flip charts in kenya," Natural Field Experiments 00256, The Field Experiments Website.
  7. Dean Karlan & Martin Valdivia, 2006. "Teaching entrepreneurship: Impact of business training on microfinance clients and institutions," Natural Field Experiments 00282, The Field Experiments Website.
  8. Tahir Andrabi & Jishnu Das & Asim Ijaz Khwaja & Tristan Zajonc, 2011. "Do Value-Added Estimates Add Value? Accounting for Learning Dynamics," American Economic Journal: Applied Economics, American Economic Association, vol. 3(3), pages 29-54, July.
  9. Gustavo J. Bobonis & Edward l Miguel & Charu Puri-Sharma, 2006. "Anaemia and School Participation," Working Papers id:337, eSocialSciences.
  10. Benjamin A. Olken, 2005. "Monitoring Corruption: Evidence from a Field Experiment in Indonesia," NBER Working Papers 11753, National Bureau of Economic Research, Inc.
  11. repec:oup:qjecon:v:122:y:2007:i:4:p:1639-1676 is not listed on IDEAS
  12. Michael Kremer, 2003. "Randomized Evaluations of Educational Programs in Developing Countries: Some Lessons," American Economic Review, American Economic Association, vol. 93(2), pages 102-106, May.
  13. Nava Ashraf & James Berry & Jesse M. Shapiro, 2007. "Can Higher Prices Stimulate Product Use? Evidence from a Field Experiment in Zambia," NBER Working Papers 13247, National Bureau of Economic Research, Inc.
  14. Esther Duflo & Rema Hanna, 2006. "Monitoring works: Getting teachers to come to school," Framed Field Experiments 00142, The Field Experiments Website.
  15. Albert Park & Meng Zhao & Paul Glewwe, 2006. "The impact of eyeglasses on the academic performance of primary school students: Evidence from a randomized trial in rural china," Natural Field Experiments 00254, The Field Experiments Website.
  16. repec:oup:qjecon:v:123:y:2008:i:4:p:1329-1372 is not listed on IDEAS
  17. repec:oup:qjecon:v:123:y:2008:i:2:p:703-745 is not listed on IDEAS
  18. Erica Field & Rohini Pande, 2008. "Repayment Frequency and Default in Microfinance: Evidence From India," Journal of the European Economic Association, MIT Press, vol. 6(2-3), pages 501-509, 04-05.
  19. repec:oup:qjecon:v:121:y:2006:i:2:p:635-672 is not listed on IDEAS
  20. Edward Miguel & Michael Kremer, 2004. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities," Econometrica, Econometric Society, vol. 72(1), pages 159-217, 01.
  21. repec:oup:qjecon:v:122:y:2007:i:3:p:1235-1264 is not listed on IDEAS
  22. Abhijit Vinayak Banerjee & Alice H. Amsden & Robert H. Bates & Jagdish Bhagwati & Angus Deaton & Nicholas Stern, 2007. "Making Aid Work," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262026155, June.
  23. repec:feb:artefa:0087 is not listed on IDEAS
Full references (including those not matched with items on IDEAS)

This item is featured on the following reading lists or Wikipedia pages:

  1. In Pursuit of Balance: Randomization in Practice in Development Field Experiments (AEJ:AE 2009) in ReplicationWiki

When requesting a correction, please mention this item's handle: RePEc:wbk:wbrwps:4752. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Roula I. Yazigi)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.