IDEAS home Printed from https://ideas.repec.org/p/hal/journl/halshs-03623394.html
   My bibliography  Save this paper

How impact evaluation methods influence the outcomes of development projects? Evidence from a meta-analysis on decentralized solar nano projects

Author

Listed:
  • Fatoumata Nankoto Cissé

    (CES - Centre d'économie de la Sorbonne - UP1 - Université Paris 1 Panthéon-Sorbonne - CNRS - Centre National de la Recherche Scientifique, I&P - Investisseurs et Partenaires)

Abstract

This study analyzes the effect of impact evaluation methodologies on the positive and negative outcomes of decentralized solar nano projects in developing countries. Data originate from the Collaborative Smart Mapping of Mini-grid Actions (CoSMMA) developed by the Foundation for Studies and Research on International Development (FERDI). This study is based on a total of 727 tested effects from 10 decentralized solar nano projects which have been measured by experimental and quasi-experimental approaches. Using a multinomial-logit regression shown that randomized and non-randomized evaluation methods have a similar probability of generating a proven favorable outcome on the sustainable development of decentralized solar nano projects. By estimating a complementary log-log model, projects are most often evaluated as successful when effects on education are tested. In addition, a discrepancy of impacts is found between randomized control trials and difference-indifference strategies in proven-unfavorable outcomes of projects. This analysis also highlights the convergence of impacts between randomization and matching techniques on projects implemented in Africa. Findings from this paper provide strong evidence for development practitioners to choose the appropriate impact assessment method.

Suggested Citation

  • Fatoumata Nankoto Cissé, 2022. "How impact evaluation methods influence the outcomes of development projects? Evidence from a meta-analysis on decentralized solar nano projects," Post-Print halshs-03623394, HAL.
  • Handle: RePEc:hal:journl:halshs-03623394
    Note: View the original document on HAL open archive server: https://shs.hal.science/halshs-03623394
    as

    Download full text from publisher

    File URL: https://shs.hal.science/halshs-03623394/document
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Angus Deaton, 2009. "Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development," Working Papers 1128, Princeton University, Woodrow Wilson School of Public and International Affairs, Center for Health and Wellbeing..
    2. Steven Glazerman & Dan M. Levy & David Myers, 2003. "Nonexperimental Versus Experimental Estimates of Earnings Impacts," The ANNALS of the American Academy of Political and Social Science, , vol. 589(1), pages 63-93, September.
    3. Angus Deaton, 2020. "Randomization in the Tropics Revisited: a Theme and Eleven Variations," NBER Working Papers 27600, National Bureau of Economic Research, Inc.
    4. Arráiz, Irani & Calero, Carla, 2015. "From Candles to Light: The Impact of Rural Electrification," IDB Publications (Working Papers) 6917, Inter-American Development Bank.
    5. Jean-Claude Berthélemy & Arnaud Millien, 2018. "Impact of Decentralized Electrification Projects on Sustainable Development: A Meta-Analysis," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) hal-01922517, HAL.
    6. Hristos Doucouliagos & Martin Paldam, 2009. "The Aid Effectiveness Literature: The Sad Results Of 40 Years Of Research," Journal of Economic Surveys, Wiley Blackwell, vol. 23(3), pages 433-461, July.
    7. Jean-Claude Berthélemy, 2019. "Challenges of decentralized electrification for economic development: lessons from experience," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) hal-02394467, HAL.
    8. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2020. "Controversies around RCT in Development. Epistemology, Ethics, and Politics," Working Papers DT/2020/15, DIAL (Développement, Institutions et Mondialisation).
    9. Martin Ravallion, 2009. "Evaluation in the Practice of Development," The World Bank Research Observer, World Bank Group, vol. 24(1), pages 29-53, March.
    10. Martin Ravallion, 2018. "Should the Randomistas (Continue to) Rule?," Working Papers 492, Center for Global Development, revised 17 Jan 2019.
    11. Rodrik, Dani, 2008. "The New Development Economics: We Shall Experiment, but How Shall We Learn?," Working Paper Series rwp08-055, Harvard University, John F. Kennedy School of Government.
    12. Labandeira, Xavier & Labeaga, José M. & Linares, Pedro & López-Otero, Xiral, 2020. "The impacts of energy efficiency policies: Meta-analysis," Energy Policy, Elsevier, vol. 147(C).
    13. Florent BEDECARRATS & Isabelle GUERIN & François ROUBAUD, 2017. "L'étalon-or des évaluations randomisées : économie politique des expérimentations aléatoires dans le domaine du développement," Working Paper 753120cd-506f-4c5f-80ed-7, Agence française de développement.
    14. repec:pri:cheawb:deaton%20instruments%20of%20development%20keynes%20lecture%202009.pdf is not listed on IDEAS
    15. Deaton, Angus & Cartwright, Nancy, 2018. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
    16. McKenzie, David, 2012. "Beyond baseline and follow-up: The case for more T in experiments," Journal of Development Economics, Elsevier, vol. 99(2), pages 210-221.
    17. Abhijit Vinayak Banerjee, 2020. "Field Experiments and the Practice of Economics," American Economic Review, American Economic Association, vol. 110(7), pages 1937-1951, July.
    18. Chauvet, Lisa & Ehrhart, Hélène, 2018. "Aid and growth: evidence from firm-level data," Journal of Development Economics, Elsevier, vol. 135(C), pages 461-477.
    19. Corduneanu-Huci, Cristina & Dorsch, Michael T. & Maarek, Paul, 2021. "The politics of experimentation: Political competition and randomized controlled trials," Journal of Comparative Economics, Elsevier, vol. 49(1), pages 1-21.
    20. repec:pri:rpdevs:instruments_of_development.pdf is not listed on IDEAS
    21. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    22. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 605-654.
    23. T. D. Stanley & Stephen B. Jarrell, 2005. "Meta‐Regression Analysis: A Quantitative Method of Literature Surveys," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 299-308, July.
    24. Michael Bamberger & Vijayendra Rao & Michael Woolcock, 2009. "Using Mixed Methods in Monitoring and Evaluation: Experiences from International Development’," Global Development Institute Working Paper Series 10709, GDI, The University of Manchester.
    25. Francis Pang & Mike Drummond & Fujian Song, 1999. "The use of meta-analysis in economic evaluation," Working Papers 173chedp, Centre for Health Economics, University of York.
    26. repec:pri:cheawb:deaton%20instruments%20of%20development%20keynes%20lecture%202009 is not listed on IDEAS
    27. Gertler, Paul J & Simone Boyce, 2003. "An Experiment in Incentive-Based Welfare: The Impact of PROGRESA on Health in Mexico," Royal Economic Society Annual Conference 2003 85, Royal Economic Society.
    28. Irani Arráiz & Carla Calero, 2015. "From Candles to Light: The Impact of Rural Electrification," IDB Publications (Working Papers) 89136, Inter-American Development Bank.
    29. repec:mpr:mprres:3694 is not listed on IDEAS
    30. Bédécarrats F (ed & Isabelle (ed.) Guérin & François (ed.) Roubaud, 2020. "Randomized control trials in the field of development : a critical perspective," Post-Print hal-03409465, HAL.
    31. Legovini, Arianna & Di Maro, Vincenzo & Piza, Caio, 2015. "Impact evaluation helps deliver development projects," Policy Research Working Paper Series 7157, The World Bank.
    32. Buddelmeyer, Hielke & Skoufias, Emmanuel, 2003. "An Evaluation of the Performance of Regression Discontinuity Design on PROGRESA," IZA Discussion Papers 827, Institute of Labor Economics (IZA).
    33. Drew B. Cameron & Anjini Mishra & Annette N. Brown, 2016. "The growth of impact evaluation for international development: how much have we learned?," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 8(1), pages 1-21, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fatoumata Nankoto Cissé, 2022. "How impact evaluation methods influence the outcomes of development projects? Evidence from a meta-analysis on decentralized solar nano projects," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-03623394, HAL.
    2. Fatoumata Nankoto Cissé, 2022. "How impact evaluation methods influence the outcomes of development projects? Evidence from a meta-analysis on decentralized solar nano projects," Documents de travail du Centre d'Economie de la Sorbonne 22008, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
    3. Tarek Azzam & Michael Bates & David Fairris, 2019. "Do Learning Communities Increase First Year College Retention? Testing Sample Selection and External Validity of Randomized Control Trials," Working Papers 202002, University of California at Riverside, Department of Economics.
    4. Vivian C. Wong & Peter M. Steiner & Kylie L. Anglin, 2018. "What Can Be Learned From Empirical Evaluations of Nonexperimental Methods?," Evaluation Review, , vol. 42(2), pages 147-175, April.
    5. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2015. "The gold standard for randomized evaluations: from discussion of method to political economy," Working Papers DT/2015/01, DIAL (Développement, Institutions et Mondialisation).
    6. Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," SciencePo Working papers Main hal-03455978, HAL.
    7. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 61, pages 3895-3962, Elsevier.
    8. Cadot, Olivier & Fernandes, Ana M. & Gourdon, Julien & Mattoo, Aaditya, 2015. "Are the benefits of export support durable? Evidence from Tunisia," Journal of International Economics, Elsevier, vol. 97(2), pages 310-324.
    9. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2019. "All that Glitters is not Gold. The Political Economy of Randomized Evaluations in Development," Development and Change, International Institute of Social Studies, vol. 50(3), pages 735-762, May.
    10. Yonatan Eyal, 2020. "Self-Assessment Variables as a Source of Information in the Evaluation of Intervention Programs: A Theoretical and Methodological Framework," SAGE Open, , vol. 10(1), pages 21582440198, January.
    11. Boris Salazar-Trujillo & Daniel Otero Robles, 2019. "La revolución empírica en economía," Apuntes del Cenes, Universidad Pedagógica y Tecnológica de Colombia, vol. 38(68), pages 15-48, July.
    12. Gustavo Canavire-Bacarreza & Luis Castro Peñarrieta & Darwin Ugarte Ontiveros, 2021. "Outliers in Semi-Parametric Estimation of Treatment Effects," Econometrics, MDPI, vol. 9(2), pages 1-32, April.
    13. Guido W. Imbens, 2010. "Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009)," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 399-423, June.
    14. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2015. "The gold standard for randomised evaluations: from discussion of method to political economics," Working Papers CEB 15-009, ULB -- Universite Libre de Bruxelles.
    15. Cook, Thomas D., 2008. ""Waiting for Life to Arrive": A history of the regression-discontinuity design in Psychology, Statistics and Economics," Journal of Econometrics, Elsevier, vol. 142(2), pages 636-654, February.
    16. Henrik Hansen & Ninja Ritter Klejnstrup & Ole Winckler Andersen, 2011. "A Comparison of Model-based and Design-based Impact Evaluations of Interventions in Developing Countries," IFRO Working Paper 2011/16, University of Copenhagen, Department of Food and Resource Economics.
    17. Enrico Nano, 2022. "Electrifying Nigeria: the Impact of Rural Access to Electricity on Kids' Schooling," IHEID Working Papers 03-2022, Economics Section, The Graduate Institute of International Studies.
    18. Mattoo, Aaditya & Cadot, Olivier & Gourdon, Julien & Fernandes, Ana Margarida, 2011. "Impact Evaluation of Trade Interventions: Paving the Way," CEPR Discussion Papers 8638, C.E.P.R. Discussion Papers.
    19. Justine Burns & Malcolm Kewsell & Rebecca Thornton, 2009. "Evaluating the Impact of Health Programmes," SALDRU Working Papers 40, Southern Africa Labour and Development Research Unit, University of Cape Town.
    20. Muller, Sean, 2014. "Randomised trials for policy: a review of the external validity of treatment effects," SALDRU Working Papers 127, Southern Africa Labour and Development Research Unit, University of Cape Town.

    More about this item

    Keywords

    Sustainable development; Decentralized electrification; Meta-analysis; Impact evaluation; Experimental methods; Quasi-experimental methods; Randomized control trials; Matching; Difference-in-difference;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:journl:halshs-03623394. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.