IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v47y2023i5p786-819.html

Intraclass Correlations Values in International Development: Evidence Across Commonly Studied Domains in sub-Saharan Africa

Author

Listed:
  • David Seidenfeld
  • Sudhanshu Handa
  • Thomas de Hoop
  • Mitchell Morey

Abstract

The sharp increase in the number of experimental studies evaluating development programs raises the need for accurate intraclass correlations (ICC) to conduct power calculations so that researchers can design studies to detect meaningful effects with sufficient statistical power. The intraclass correlation is an important parameter for determining the statistical power of cluster-randomized trials. The parameter is rarely available to researchers planning a study until after the design is set and data are already collected. This paper takes an important step towards helping researchers working in sub-Saharan Africa to accurately estimate appropriate sample sizes for their clustered RCTs. The study draws from rich data sets in Kenya, Malawi, Zambia, and Zimbabwe. We present ICCs for a wide range of domains common for development research. Our results suggest that ICCs for commonly studied indicators in sub-Saharan Africa are lower than is often assumed in power calculations. ICC values are especially low for indicators associated with child nutrition and food security, suggesting that cluster-RCTs might be a viable design even when faced with limited budgets because sample size requirements are not much different from an individual random assignment design.

Suggested Citation

  • David Seidenfeld & Sudhanshu Handa & Thomas de Hoop & Mitchell Morey, 2023. "Intraclass Correlations Values in International Development: Evidence Across Commonly Studied Domains in sub-Saharan Africa," Evaluation Review, , vol. 47(5), pages 786-819, October.
  • Handle: RePEc:sae:evarev:v:47:y:2023:i:5:p:786-819
    DOI: 10.1177/0193841X231154714
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X231154714
    Download Restriction: no

    File URL: https://libkey.io/10.1177/0193841X231154714?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    2. Ward Heneveld & Helen Craig, 1996. "Schools Count : World Bank Project Designs and the Quality of African Primary Education," World Bank Publications - Reports 9974, The World Bank Group.
    3. A. Colin Cameron & Douglas L. Miller, 2015. "A Practitioner’s Guide to Cluster-Robust Inference," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 317-372.
    4. Larry V. Hedges & E. C. Hedberg, 2013. "Intraclass Correlations and Covariate Outcome Correlations for Planning Two- and Three-Level Cluster-Randomized Experiments in Education," Evaluation Review, , vol. 37(6), pages 445-489, December.
    5. John P. A. Ioannidis & T. D. Stanley & Hristos Doucouliagos, 2017. "The Power of Bias in Economics Research," Economic Journal, Royal Economic Society, vol. 127(605), pages 236-265, October.
    6. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 61, pages 3895-3962, Elsevier.
    7. Abhijit Banerjee & Dean Karlan & Jonathan Zinman, 2015. "Six Randomized Evaluations of Microcredit: Introduction and Further Steps," American Economic Journal: Applied Economics, American Economic Association, vol. 7(1), pages 1-21, January.
    8. Drew B. Cameron & Anjini Mishra & Annette N. Brown, 2016. "The growth of impact evaluation for international development: how much have we learned?," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 8(1), pages 1-21, March.
    9. Alberto Abadie & Susan Athey & Guido W Imbens & Jeffrey M Wooldridge, 2023. "When Should You Adjust Standard Errors for Clustering?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 138(1), pages 1-35.
    10. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Steven Glazerman & Larissa Campuzano & Nancy Murray, 2025. "Education Experiments in Latin America: Empirical Evidence to Guide Evaluation Design," Evaluation Review, , vol. 49(1), pages 115-146, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. repec:osf:osfxxx:nwp8k_v1 is not listed on IDEAS
    2. Brox, Enzo & Krieger, Tommy, 2022. "Birthplace diversity and team performance," Labour Economics, Elsevier, vol. 79(C).
    3. Jushan Bai & Sung Hoon Choi & Yuan Liao, 2021. "Feasible generalized least squares for panel data with cross-sectional and serial correlations," Empirical Economics, Springer, vol. 60(1), pages 309-326, January.
    4. Matthias Breuer & Ed Dehaan, 2024. "Using and Interpreting Fixed Effects Models," Journal of Accounting Research, John Wiley & Sons, Ltd., vol. 62(4), pages 1183-1226, September.
    5. Seonho Shin, 2021. "Were they a shock or an opportunity?: The heterogeneous impacts of the 9/11 attacks on refugees as job seekers—a nonlinear multi-level approach," Empirical Economics, Springer, vol. 61(5), pages 2827-2864, November.
    6. Masselus, Lise & Petrik, Christina & Ankel-Peters, Jörg, 2024. "Lost in the design space? Construct validity in the microfinance literature," Ruhr Economic Papers 1097, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    7. Zuzana Irsova & Pedro R. D. Bom & Tomas Havranek & Heiko Rachinger, 2025. "Spurious precision in meta-analysis of observational research," Nature Communications, Nature, vol. 16(1), pages 1-12, December.
    8. Nobel Prize Committee, 2019. "Understanding development and poverty alleviation," Nobel Prize in Economics documents 2019-2, Nobel Prize Committee.
    9. Dahal, Mahesh & Fiala, Nathan, 2018. "What do we know about the impact of microfinance? The problems of power and precision," Ruhr Economic Papers 756, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    10. Yantao Wen & Yuanfei Kang & Yafeng Qin & Jeffrey C. Kennedy, 2021. "Use of Derivative and Firm Performance: Evidence from the Chinese Shenzhen Stock Exchange," JRFM, MDPI, vol. 14(2), pages 1-22, February.
    11. Aleksey Oshchepkov & Anna Shirokanova, 2020. "Multilevel Modeling For Economists: Why, When And How," HSE Working papers WP BRP 233/EC/2020, National Research University Higher School of Economics.
    12. Ejermo, Olof & Sofer, Yotam, 2024. "When colleges graduate: Micro-level effects on publications and scientific organization," Research Policy, Elsevier, vol. 53(6).
    13. Jeffrey D. Michler & Anna Josephson, 2022. "Recent developments in inference: practicalities for applied economics," Chapters, in: A Modern Guide to Food Economics, chapter 11, pages 235-268, Edward Elgar Publishing.
    14. Wagner, Johannes & Merner, Sara & Innocenti, Stefania & Geling, Alinta & Hope, Rob, 2025. "Can solar water kiosks generate sustainable revenue streams for rural water services?," World Development, Elsevier, vol. 185(C).
    15. Alex Eble & Peter Boone & Diana Elbourne, 2017. "On Minimizing the Risk of Bias in Randomized Controlled Trials in Economics," The World Bank Economic Review, World Bank, vol. 31(3), pages 687-707.
    16. Christopher Snyder & Ran Zhuo, 2018. "Sniff Tests as a Screen in the Publication Process: Throwing out the Wheat with the Chaff," NBER Working Papers 25058, National Bureau of Economic Research, Inc.
    17. Chowdhury, Shyamal & Hasan, Syed & Sharma, Uttam, 2024. "The Role of Trainee Selection in the Effectiveness of Vocational Training: Evidence from a Randomized Controlled Trial in Nepal," IZA Discussion Papers 16705, IZA Network @ LISER.
    18. Bai, Jushan & Choi, Sung Hoon & Liao, Yuan, 2024. "Standard errors for panel data models with unknown clusters," Journal of Econometrics, Elsevier, vol. 240(2).
    19. Millimet, Daniel L. & Bellemare, Marc, 2023. "Fixed Effects and Causal Inference," IZA Discussion Papers 16202, IZA Network @ LISER.
    20. Eduard Marinov, 2019. "The 2019 Nobel Prize in Economics," Economic Thought journal, Bulgarian Academy of Sciences - Economic Research Institute, issue 6, pages 78-116.
    21. Dan Zhou & Xue Li & Yao Yao & George S. Chen, 2025. "“If you build it, will they come?”—Did city commercial bank development shape the spatial evolution of foreign direct investment in China?," The Annals of Regional Science, Springer;Western Regional Science Association, vol. 74(2), pages 1-30, June.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:47:y:2023:i:5:p:786-819. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.