IDEAS home Printed from https://ideas.repec.org/p/ess/wpaper/id2187.html
   My bibliography  Save this paper

Rough Guide to Impact Evaluation of Environmental and Development Programs

Author

Listed:
  • Subhrendu K Pattanayak

Abstract

This paper is a “rough guide” for evaluation of programs, projects and policies in the environment and development arena. First, a general overview of the what, how, and why of program evaluation, with particular emphasis on the role of control groups, pre and post measurement, and covariate data to define counterfactual scenarios (including formal definition of all terms) are provided. Second, a detailed review with examples of the four main methods for evaluation – randomized experiments, natural experiments, matching methods, and panel-based DID estimators with a description of the pros and cons of each method is given. Finally, the econometric evaluations within the broader context is placed– how can we move beyond estimation of average treatment effects; what do we do under time, resource and data constraints; and when and where should we rely on theory-based evaluations.

Suggested Citation

  • Subhrendu K Pattanayak, 2009. "Rough Guide to Impact Evaluation of Environmental and Development Programs," Working Papers id:2187, eSocialSciences.
    • Handle: RePEc:ess:wpaper:id:2187
    as

    Download full text from publisher

    File URL: http://www.eSocialSciences.com/data/articles/Document11982009570.2346613.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Christina Paxson & Norbert R. Schady, 2002. "The Allocation and Impact of Social Funds: Spending on School Infrastructure in Peru," The World Bank Economic Review, World Bank, vol. 16(2), pages 297-319, August.
    2. Jalan, Jyotsna & Ravallion, Martin, 2003. "Does piped water reduce diarrhea for children in rural India?," Journal of Econometrics, Elsevier, vol. 112(1), pages 153-173, January.
    3. Ferraro, Paul J. & McIntosh, Craig & Ospina, Monica, 2007. "The effectiveness of the US endangered species act: An econometric analysis using matching methods," Journal of Environmental Economics and Management, Elsevier, vol. 54(3), pages 245-261, November.
    4. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    5. Menno Pradhan & Laura B. Rawlings, 2002. "The Impact and Targeting of Social Infrastructure Investments: Lessons from the Nicaraguan Social Fund," The World Bank Economic Review, World Bank, vol. 16(2), pages 275-295, August.
    6. Kenneth Y. Chay & Michael Greenstone, 2003. "The Impact of Air Pollution on Infant Mortality: Evidence from Geographic Variation in Pollution Shocks Induced by a Recession," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(3), pages 1121-1167.
    7. Edmonds, Eric V., 2002. "Government-initiated community resource management and local resource extraction from Nepal's forests," Journal of Development Economics, Elsevier, vol. 68(1), pages 89-115, June.
    8. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 61, pages 3895-3962, Elsevier.
    9. Kenneth Y. Chay & Michael Greenstone, 2005. "Does Air Quality Matter? Evidence from the Housing Market," Journal of Political Economy, University of Chicago Press, vol. 113(2), pages 376-424, April.
    10. Halbert White, 2004. "New Perspectives in Econometric Theory," Books, Edward Elgar Publishing, number 3245.
    11. John Newman & Menno Pradhan & Laura B. Rawlings & Geert Ridder & Ramiro Coa & Jose Luis Evia, 2002. "An Impact Evaluation of Education, Health, and Water Supply Investments by the Bolivian Social Investment Fund," The World Bank Economic Review, World Bank, vol. 16(2), pages 241-274, August.
    12. Jalan, Jyotsna & Somanathan, E., 2008. "The importance of being informed: Experimental evidence on demand for environmental quality," Journal of Development Economics, Elsevier, vol. 87(1), pages 14-28, August.
    13. Judy L. Baker, 2000. "Evaluating the Impact of Development Projects on Poverty : A Handbook for Practitioners," World Bank Publications - Books, The World Bank Group, number 13949, December.
    14. Isabelle Huault & V. Perret & S. Charreire-Petit, 2007. "Management," Post-Print halshs-00337676, HAL.
    15. Frondel, Manuel & Schmidt, Christoph M., 2005. "Evaluating environmental programs: The perspective of modern evaluation research," Ecological Economics, Elsevier, vol. 55(4), pages 515-526, December.
    16. Operations Evaluation Department, 2004. "Monitoring and Evaluation," World Bank Publications - Books, The World Bank Group, number 23975, December.
    17. Robert S. Chase, 2002. "Supporting Communities in Transition: The Impact of the Armenian Social Investment Fund," The World Bank Economic Review, World Bank, vol. 16(2), pages 219-240, August.
    18. Leach, Gerald, 1992. "The energy transition," Energy Policy, Elsevier, vol. 20(2), pages 116-123, February.
    19. Elizabeth Frankenberg & Douglas McKee & Duncan Thomas, 2005. "Health consequences of forest fires in Indonesia," Demography, Springer;Population Association of America (PAA), vol. 42(1), pages 109-129, February.
    20. Laura B. Rawlings & Lynne Sherburne-Benz & Julie van Domelen, 2004. "Evaluating Social Funds : A Cross-Country Analysis of Community Investments," World Bank Publications - Books, The World Bank Group, number 15057, December.
    21. Narain, Urvashi & Gupta, Shreekant & van 't Veld, Klaas, 2008. "Poverty and resource dependence in rural India," Ecological Economics, Elsevier, vol. 66(1), pages 161-176, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Brooks, N. & Bhojvaid, V. & Jeuland, M.A. & Lewis, J.J. & Patange, O. & Pattanayak, S.K., 2016. "How much do alternative cookstoves reduce biomass fuel use? Evidence from North India," Resource and Energy Economics, Elsevier, vol. 43(C), pages 153-171.
    2. Miteva, Daniela A. & Murray, Brian C. & Pattanayak, Subhrendu K., 2015. "Do protected areas reduce blue carbon emissions? A quasi-experimental evaluation of mangroves in Indonesia," Ecological Economics, Elsevier, vol. 119(C), pages 127-135.
    3. Reed, James & van Vianen, Josh & Foli, Samson & Clendenning, Jessica & Yang, Kevin & MacDonald, Margaret & Petrokofsky, Gillian & Padoch, Christine & Sunderland, Terry, 2017. "Trees for life: The ecosystem service contribution of trees to food production and livelihoods in the tropics," Forest Policy and Economics, Elsevier, vol. 84(C), pages 62-71.
    4. Patnaik, Unmesh & Das, Prasun Kumar, 2017. "Do Development Interventions Confer Adaptive Capacity? Insights from Rural India," World Development, Elsevier, vol. 97(C), pages 298-312.
    5. Bauch, Simone C. & Sills, Erin O. & Pattanayak, Subhrendu K., 2014. "Have We Managed to Integrate Conservation and Development? ICDP Impacts in the Brazilian Amazon," World Development, Elsevier, vol. 64(S1), pages 135-148.
    6. Gebrehiwot, Desta Brhanu, 2015. "Impact of Weather Index Insurance on Household Demand for Fertilizer in Tigray Region," MPRA Paper 94410, University Library of Munich, Germany.
    7. Mohebalian, Phillip M. & Aguilar, Francisco X., 2016. "Additionality and design of forest conservation programs: Insights from Ecuador's Socio Bosque Program," Forest Policy and Economics, Elsevier, vol. 71(C), pages 103-114.
    8. Kohlin, Gunnar & Sills, Erin O. & Pattanayak, Subhrendu K. & Wilfong, Christopher, 2011. "Energy, gender and development: what are the linkages ? where is the evidence ?," Policy Research Working Paper Series 5800, The World Bank.
    9. Patrick Bottazzi & Mirko S. Winkler & Sébastien Boillat & Abdoulaye Diagne & Mashoudou Maman Chabi Sika & Arsène Kpangon & Salimata Faye & Chinwe Ifejika Speranza, 2018. "Measuring Subjective Flood Resilience in Suburban Dakar: A Before–After Evaluation of the “Live with Water” Project," Sustainability, MDPI, vol. 10(7), pages 1-14, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ravallion, Martin, 2008. "Evaluating Anti-Poverty Programs," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 59, pages 3787-3846, Elsevier.
    2. Araujo, M. Caridad & Ferreira, Francisco H.G. & Lanjouw, Peter & Özler, Berk, 2008. "Local inequality and project choice: Theory and evidence from Ecuador," Journal of Public Economics, Elsevier, vol. 92(5-6), pages 1022-1046, June.
    3. Eric W. Djimeu, 2014. "Does social action fund promote schooling in conflict affected countries? Mixed evidence from Angola," HiCN Working Papers 189, Households in Conflict Network.
    4. Ghazala Mansuri, 2004. "Community-Based and -Driven Development: A Critical Review," The World Bank Research Observer, World Bank, vol. 19(1), pages 1-39.
    5. Vijayendra Rao & Ana Maria Ibanez, 2005. "The Social Impact of Social Funds in Jamaica: A 'Participatory Econometric' Analysis of Targeting, Collective Action, and Participation in Community-Driven Development," Journal of Development Studies, Taylor & Francis Journals, vol. 41(5), pages 788-838.
    6. World Bank, 2008. "Community-Driven Approaches in Lao PDR : Moving Beyond Service Delivery - Summary Overview," World Bank Publications - Reports 6163, The World Bank Group.
    7. World Bank, 2008. "Community-Driven Approaches in Lao PDR : Moving Beyond Service Delivery, Volume 2. Main Report," World Bank Publications - Reports 7973, The World Bank Group.
    8. Greenstone, Michael & Gayer, Ted, 2009. "Quasi-experimental and experimental approaches to environmental economics," Journal of Environmental Economics and Management, Elsevier, vol. 57(1), pages 21-44, January.
    9. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    10. Harounan Kazianga & Leigh Linden & Ali Protik & Matt Sloan, 2016. "The Medium-Term Impacts of Girl-Friendly Schools: Seven-Year Evidence from School Construction in Burkina Faso," Development Working Papers 406, Centro Studi Luca d'Agliano, University of Milano, revised 11 Nov 2016.
    11. Galdo, Virgilio & Briceño, Bertha, 2005. "Evaluating the Impact on Child Mortality of a Water Supply and Sewerage Expansion in Quito: Is Water Enough?," IDB Publications (Working Papers) 2833, Inter-American Development Bank.
    12. Park, Albert & Wang, Sangui, 2010. "Community-based development and poverty alleviation: An evaluation of China's poor village investment program," Journal of Public Economics, Elsevier, vol. 94(9-10), pages 790-799, October.
    13. Do, Quy-Toan & Joshi, Shareen & Stolper, Samuel, 2018. "Can environmental policy reduce infant mortality? Evidence from the Ganga Pollution Cases," Journal of Development Economics, Elsevier, vol. 133(C), pages 306-325.
    14. Antonio Estache, 2010. "A survey of impact evaluations of infrastructure projects, programs and policies," Working Papers ECARES 2010_005, ULB -- Universite Libre de Bruxelles.
    15. Luis Marcano, 2005. "Atacando Pobreza: Evaluación del Programa Fondo de Inversión Social de Panamá," OVE Working Papers 0205, Inter-American Development Bank, Office of Evaluation and Oversight (OVE).
    16. Justine Burns & Malcolm Kewsell & Rebecca Thornton, 2009. "Evaluating the Impact of Health Programmes," SALDRU Working Papers 40, Southern Africa Labour and Development Research Unit, University of Cape Town.
    17. Wagstaff, Adam & Pradhan, Menno, 2005. "Health insurance impacts on health and nonmedical consumption in a developing country," Policy Research Working Paper Series 3563, The World Bank.
    18. Djimeu, Eric W., 2014. "The impact of social action funds on child health in a conflict affected country: Evidence from Angola," Social Science & Medicine, Elsevier, vol. 106(C), pages 35-42.
    19. Shapiro, Joseph & Trevino, Jorge Moreno, 2004. "Compensatory education for disadvantaged Mexican students : an impact evaluation using propensity score matching," Policy Research Working Paper Series 3334, The World Bank.
    20. Lay, Jann, 2010. "MDG Achievements, Determinants, and Resource Needs: What Has Been Learnt?," GIGA Working Papers 137, GIGA German Institute of Global and Area Studies.

    More about this item

    Keywords

    evaluation; environment; development area; econometric; natural and quasi experiments; resource; data constraints; data; theory; DID estimators; south asia;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ess:wpaper:id:2187. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Padma Prakash (email available below). General contact details of provider: http://www.esocialsciences.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.