IDEAS home Printed from https://ideas.repec.org/p/wbk/wbrwps/4784.html
   My bibliography  Save this paper

Almost Random: Evaluating a Large-Scale Randomized Nutrition Program in the Presence of Crossover

Author

Listed:
  • Linnemayr, Sebastian

    (Harvard University)

  • Alderman, Harold

    (The World Bank)

Abstract

Large-scale randomized interventions have the potential to uncover the causal effect of programs applying to a large population, thereby improving on the insights gained from currently dominant smaller randomized studies. However, the external validity gained through larger interventions typically implies less supervision and often comes at the cost of some deviation from the randomization plan. This paper investigates the impact of the Nutrition Enhancement Program, which aims to improve child nutrition in Senegal based on a large-scale randomized community intervention. The analysis explicitly deals with deviation from the planned treatment and suggests approaches for combining ex-post adjustments such as propensity score matching with the randomized treatment plan. The authors do not detect a strong overall program impact on the outcome measure of weight-for-age based on planned treatment status, but do find an impact on the youngest children. Moreover, the project impact is clearer when the analysis considers treatment crossover using alternative estimators of two-stage least-squares and propensity score matching. The findings underscore the importance of addressing the shortcomings of large-scale randomization interventions in a systematic manner in order to understand the selection process that can guide further implementation of such projects, as well as to expose the true, causal effect of such programs.

Suggested Citation

  • Linnemayr, Sebastian & Alderman, Harold, 2008. "Almost Random: Evaluating a Large-Scale Randomized Nutrition Program in the Presence of Crossover," Policy Research Working Paper Series 4784, The World Bank.
  • Handle: RePEc:wbk:wbrwps:4784
    as

    Download full text from publisher

    File URL: http://www-wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2008/12/09/000158349_20081209104903/Rendered/PDF/WPS4784.pdf
    File Function: Full text
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Harold Alderman & John Hoddinott & Bill Kinsey, 2006. "Long term consequences of early childhood malnutrition," Oxford Economic Papers, Oxford University Press, vol. 58(3), pages 450-474, July.
    2. Kenneth Y. Chay & Patrick J. McEwan & Miguel Urquiola, 2005. "The Central Role of Noise in Evaluating Interventions That Use Test Scores to Rank Schools," American Economic Review, American Economic Association, vol. 95(4), pages 1237-1258, September.
    3. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 61, pages 3895-3962, Elsevier.
    4. Susan Horton & Meera Shekar & Christine McDonald & Ajay Mahal & Jana Krystene Brooks, 2010. "Scaling Up Nutrition : What Will it Cost?," World Bank Publications - Books, The World Bank Group, number 2685.
    5. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    6. Joshua Angrist & Eric Bettinger & Erik Bloom & Elizabeth King & Michael Kremer, 2002. "Vouchers for Private Schooling in Colombia: Evidence from a Randomized Natural Experiment," American Economic Review, American Economic Association, vol. 92(5), pages 1535-1558, December.
    7. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, July.
    8. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    9. Ravallion, Martin, 2008. "Evaluating Anti-Poverty Programs," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 59, pages 3787-3846, Elsevier.
    10. Guido W. Imbens, 2010. "Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009)," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 399-423, June.
    11. Elizabeth M. King & Jere R. Behrman, 2009. "Timing and Duration of Exposure in Evaluations of Social Programs," The World Bank Research Observer, World Bank Group, vol. 24(1), pages 55-82, February.
    12. Jere R. Behrman & Mark R. Rosenzweig, 2004. "Returns to Birthweight," The Review of Economics and Statistics, MIT Press, vol. 86(2), pages 586-601, May.
    13. Strauss, John, 1990. "Households, Communities, and Preschool Children's Nutrition Outcomes: Evidence from Rural Cote d'Ivoire," Economic Development and Cultural Change, University of Chicago Press, vol. 38(2), pages 231-261, January.
    14. Angrist, Joshua D. & Krueger, Alan B., 1999. "Empirical strategies in labor economics," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 23, pages 1277-1366, Elsevier.
    15. O. Ashenfelter & D. Card (ed.), 1999. "Handbook of Labor Economics," Handbook of Labor Economics, Elsevier, edition 1, volume 3, number 3.
    16. Behrman, Jere R. & Skoufias, Emmanuel, 2004. "Correlates and determinants of child anthropometrics in Latin America: background and overview of the symposium," Economics & Human Biology, Elsevier, vol. 2(3), pages 335-351, December.
    17. Alderman, Harold, 2007. "Improving Nutrition through Community Growth Promotion: Longitudinal Study of the Nutrition and Early Child Development Program in Uganda," World Development, Elsevier, vol. 35(8), pages 1376-1389, August.
    18. Bjorklund, Anders & Moffitt, Robert, 1987. "The Estimation of Wage Gains and Welfare Gains in Self-selection," The Review of Economics and Statistics, MIT Press, vol. 69(1), pages 42-49, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Douglas Almond & Janet Currie & Valentina Duque, 2018. "Childhood Circumstances and Adult Outcomes: Act II," Journal of Economic Literature, American Economic Association, vol. 56(4), pages 1360-1446, December.
    2. Asfaw, Abraham Abebe, 2018. "The distributional effect of investment in early childhood nutrition: A panel quantile approach," World Development, Elsevier, vol. 110(C), pages 63-74.
    3. Giles, John & Satriawan, Elan, 2015. "Protecting child nutritional status in the aftermath of a financial crisis: Evidence from Indonesia," Journal of Development Economics, Elsevier, vol. 114(C), pages 97-106.
    4. Janet Currie & Tom Vogl, 2013. "Early-Life Health and Adult Circumstance in Developing Countries," Annual Review of Economics, Annual Reviews, vol. 5(1), pages 1-36, May.
    5. Fitzsimons, Emla & Malde, Bansi & Mesnard, Alice & Vera-Hernández, Marcos, 2016. "Nutrition, information and household behavior: Experimental evidence from Malawi," Journal of Development Economics, Elsevier, vol. 122(C), pages 113-126.
    6. Harold Alderman & Derek Headey, 2018. "The timing of growth faltering has important implications for observational analyses of the underlying determinants of nutrition outcomes," PLOS ONE, Public Library of Science, vol. 13(4), pages 1-16, April.
    7. repec:pri:rpdevs:currie_vogl_ar is not listed on IDEAS
    8. Araújo, Daniel & Carrillo, Bladimir & Sampaio, Breno, 2021. "The Long-Run Economic Consequences of Iodine Supplementation," Journal of Health Economics, Elsevier, vol. 79(C).
    9. Ecker, Olivier & Mabiso, Athur & Kennedy, Adam & Diao, Xinshen 22905, 2011. "Making agriculture pro-nutrition: Opportunities in Tanzania," IFPRI discussion papers 1124, International Food Policy Research Institute (IFPRI).
    10. Marian Meller & Stephan Litschig, 2014. "Saving Lives: Evidence from a Conditional Food Supplementation Program," Journal of Human Resources, University of Wisconsin Press, vol. 49(4), pages 1014-1052.
    11. Kelly A. Davidson & Jaclyn D. Kropp & Conner Mullally & Md. Wakilur Rahman, 2021. "Can Simple Nudges and Workshops Improve Diet Quality? Evidence from a Randomized Trial in Bangladesh," American Journal of Agricultural Economics, John Wiley & Sons, vol. 103(1), pages 253-274, January.
    12. Premand, Patrick & Barry, Oumar, 2022. "Behavioral change promotion, cash transfers and early childhood development: Experimental evidence from a government program in a low-income setting," Journal of Development Economics, Elsevier, vol. 158(C).
    13. Mark E. McGovern, 2019. "How much does birth weight matter for child health in developing countries? Estimates from siblings and twins," Health Economics, John Wiley & Sons, Ltd., vol. 28(1), pages 3-22, January.
    14. Siddhartha Bandyopadhyay & Sanjukta Sarkar & Rudra Sensarma, 2021. "Does Access to Key Household Resources Help in Reducing Violence against Women?," Discussion Papers 21-09, Department of Economics, University of Birmingham.
    15. Fitzsimons, Emla & Malde, Bansi & Mesnard, Alice & Vera-Hernández, Marcos, 2012. "Household Responses to Information on Child Nutrition: Experimental Evidence from Malawi," CEPR Discussion Papers 8915, C.E.P.R. Discussion Papers.
    16. Davidson, Kelly A. & Kropp, Jaclyn D. & Mullally, Conner C. & Rahman, M. Wakilur, 2018. "Behavioral Nudges and Nutrition Education in Bangladesh: Experimental Evidence Comparing Food Choices in a Lab Setting to Decisions at Home," 2018 Annual Meeting, August 5-7, Washington, D.C. 274134, Agricultural and Applied Economics Association.
    17. Alderman, Harold & Gilligan, Daniel O. & Leight, Jessica & Mulford, Michael & Tambet, Heleene, 2022. "The role of poultry transfers in diet diversity: A cluster randomized intent to treat analysis," Food Policy, Elsevier, vol. 107(C).
    18. Álvarez, Begoña & Vera-Hernández, Marcos, 2013. "Exploiting subjective information to understand impoverished children's use of health care," Journal of Health Economics, Elsevier, vol. 32(6), pages 1194-1204.
    19. Karen Clay & Ethan J. Schmick & Werner Troesken, 2020. "The Boll Weevil’s Impact on Racial Income Gaps in the Early Twentieth Century," NBER Working Papers 27101, National Bureau of Economic Research, Inc.
    20. Bonner, Suzanne & Sarkar, Dipanwita, 2018. "The quality-quantity trade-off among Australian children," Economic Modelling, Elsevier, vol. 70(C), pages 383-389.
    21. Bonou, Alice & Diagne, Aliou & Biaou, Gauthier, 2013. "Agricultural technology adoption and rice varietal diversity: A Local Average Treatment Effect (LATE) Approach for rural Benin," 2013 Fourth International Conference, September 22-25, 2013, Hammamet, Tunisia 158482, African Association of Agricultural Economists (AAAE).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    2. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 61, pages 3895-3962, Elsevier.
    3. Schultz, T. Paul, 2010. "Population and Health Policies," Handbook of Development Economics, in: Dani Rodrik & Mark Rosenzweig (ed.), Handbook of Development Economics, edition 1, volume 5, chapter 0, pages 4785-4881, Elsevier.
    4. Committee, Nobel Prize, 2021. "Answering causal questions using observational data," Nobel Prize in Economics documents 2021-2, Nobel Prize Committee.
    5. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    6. Pedro Carneiro & James J. Heckman & Edward J. Vytlacil, 2011. "Estimating Marginal Returns to Education," American Economic Review, American Economic Association, vol. 101(6), pages 2754-2781, October.
    7. Ferraro, Paul J. & Miranda, Juan José, 2014. "The performance of non-experimental designs in the evaluation of environmental programs: A design-replication study using a large-scale randomized experiment as a benchmark," Journal of Economic Behavior & Organization, Elsevier, vol. 107(PA), pages 344-365.
    8. Sokbae Lee & Yoon-Jae Whang, 2009. "Nonparametric Tests of Conditional Treatment Effects," Cowles Foundation Discussion Papers 1740, Cowles Foundation for Research in Economics, Yale University.
    9. James J. Heckman, 2008. "The Principles Underlying Evaluation Estimators with an Application to Matching," Annals of Economics and Statistics, GENES, issue 91-92, pages 9-73.
    10. Flores-Lagunes, Alfonso & Gonzalez, Arturo & Neumann, Todd C., 2007. "Estimating the Effects of Length of Exposure to a Training Program: The Case of Job Corps," IZA Discussion Papers 2846, Institute of Labor Economics (IZA).
    11. Baum-Snow, Nathaniel & Ferreira, Fernando, 2015. "Causal Inference in Urban and Regional Economics," Handbook of Regional and Urban Economics, in: Gilles Duranton & J. V. Henderson & William C. Strange (ed.), Handbook of Regional and Urban Economics, edition 1, volume 5, chapter 0, pages 3-68, Elsevier.
    12. Angus Deaton, 2009. "Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development," Working Papers 1128, Princeton University, Woodrow Wilson School of Public and International Affairs, Center for Health and Wellbeing..
    13. Behrman, Jere R., 2010. "Investment in Education Inputs and Incentives," Handbook of Development Economics, in: Dani Rodrik & Mark Rosenzweig (ed.), Handbook of Development Economics, edition 1, volume 5, chapter 0, pages 4883-4975, Elsevier.
    14. Chakravarty, Shubha & Lundberg, Mattias & Nikolov, Plamen & Zenker, Juliane, 2019. "Vocational training programs and youth labor market outcomes: Evidence from Nepal," Journal of Development Economics, Elsevier, vol. 136(C), pages 71-110.
    15. Rajeev Dehejia, 2013. "The Porous Dialectic: Experimental and Non-Experimental Methods in Development Economics," WIDER Working Paper Series wp-2013-011, World Institute for Development Economic Research (UNU-WIDER).
    16. Dehejia Rajeev, 2015. "Experimental and Non-Experimental Methods in Development Economics: A Porous Dialectic," Journal of Globalization and Development, De Gruyter, vol. 6(1), pages 47-69, June.
    17. Dehejia, Rajeev, 2013. "The Porous Dialectic: Experimental and Non-Experimental Methods in Development Economics," WIDER Working Paper Series 011, World Institute for Development Economic Research (UNU-WIDER).
    18. Asadul Islam Author-X-Name-Asadul, 2008. "Who Benefits From Microfinance? The Impact Evaluation Of Large Scale Programs In Bangladesh," Monash Economics Working Papers 29/08, Monash University, Department of Economics.
    19. Heckman, James J., 2010. "The Assumptions Underlying Evaluation Estimators," Brazilian Review of Econometrics, Sociedade Brasileira de Econometria - SBE, vol. 30(2), December.
    20. van de Walle, Dominique & Mu, Ren, 2007. "Fungibility and the flypaper effect of project aid: Micro-evidence for Vietnam," Journal of Development Economics, Elsevier, vol. 84(2), pages 667-685, November.

    More about this item

    Keywords

    Nutrition; Impact evaluation;

    JEL classification:

    • I12 - Health, Education, and Welfare - - Health - - - Health Behavior
    • O12 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Microeconomic Analyses of Economic Development

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wbk:wbrwps:4784. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/dvewbus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Roula I. Yazigi (email available below). General contact details of provider: https://edirc.repec.org/data/dvewbus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.