IDEAS home Printed from https://ideas.repec.org/p/pri/rpdevs/hammer_its_all_about_me.pdf.html
   My bibliography  Save this paper

It's All About MeE: Using Structured Experiential Learning ('e') to Crawl the Design Space

Author

Listed:
  • Lant Pritchett

    (Harvard Kennedy School and Center for Global Development)

  • Salimah Samji

    (Harvard Kennedy School)

  • Jeffrey Hammer

    (Princeton University)

Abstract

Organizations that fund development projects whether they be governments, multi-laterals, bilateral agencies, or NGOs have to make hard decisions about what to fund. In this decision there is an inherent tension between funding activities that have solid evidence about effectiveness and funding innovative activities that promise even greater effectiveness but are untested. Evidence based approaches that promote greater use of Rigorous Impact Evaluations (including randomized control trials) and evidence from those evaluations in policy and programming have added more rigor to the E (evaluation) in traditional M&E. Here we extend the basic idea of rigorous impact evaluation?the use of a valid counter-factual to make judgments about causality to evaluate project design and implementation. This adds a new learning component of experiential learning or a little e to the M&RIE so that instead of just M&E development projects are all about MeE. Structured experiential learning allows implementing agencies to actively and rigorously search across alternative project designs using the monitoring data that provides real time performance information with direct feedback into project design and implementation. The key insight is that within- project variations can serve as their own counter-factual which dramatically reduces the incremental cost of evaluation and increases the usefulness of evaluation to implementing agencies. The right combination of MeE provides for rigorous learning while the providing needed space for innovation.

Suggested Citation

  • Lant Pritchett & Salimah Samji & Jeffrey Hammer, 2012. "It's All About MeE: Using Structured Experiential Learning ('e') to Crawl the Design Space," Working Papers 1399, Princeton University, Woodrow Wilson School of Public and International Affairs, Research Program in Development Studies..
  • Handle: RePEc:pri:rpdevs:hammer_its_all_about_me.pdf
    as

    Download full text from publisher

    File URL: https://rpds.princeton.edu/sites/rpds/files/media/hammer_its_all_about_me.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Hammer, Jeffrey S, 1997. "Economic Analysis for Health Projects," The World Bank Research Observer, World Bank, vol. 12(1), pages 47-71, February.
    2. Baird, Sarah & McIntosh, Craig & Ozler, Berk, 2009. "Designing cost-effective cash transfer programs to boost schooling among young women in Sub-Saharan Africa," Policy Research Working Paper Series 5090, The World Bank.
    3. Ravallion Martin, 2015. "On the Implications of Essential Heterogeneity for Estimating Causal Impacts Using Social Experiments," Journal of Econometric Methods, De Gruyter, vol. 4(1), pages 1-7, January.
    4. Angus Deaton, 2009. "Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development," Working Papers 1128, Princeton University, Woodrow Wilson School of Public and International Affairs, Center for Health and Wellbeing..
    5. Elizabeth M. King & Jere R. Behrman, 2009. "Timing and Duration of Exposure in Evaluations of Social Programs," The World Bank Research Observer, World Bank, vol. 24(1), pages 55-82, February.
    6. Andrews, Matt & Pritchett, Lant & Woolcock, Michael, 2013. "Escaping Capability Traps Through Problem Driven Iterative Adaptation (PDIA)," World Development, Elsevier, vol. 51(C), pages 234-244.
    7. White, Howard, 2006. "Impact evaluation: the experience of the Independent Evaluation Group of the World Bank," MPRA Paper 1111, University Library of Munich, Germany.
    8. Lant Pritchett & Michael Woolcock & Matt Andrews, 2013. "Looking Like a State: Techniques of Persistent Failure in State Capability for Implementation," Journal of Development Studies, Taylor & Francis Journals, vol. 49(1), pages 1-18, January.
    9. Abhijit Banerjee & Raghabendra Chattopadhyay & Esther Duflo & Daniel Keniston & Nina Singh, 2021. "Improving Police Performance in Rajasthan, India: Experimental Evidence on Incentives, Managerial Autonomy, and Training," American Economic Journal: Economic Policy, American Economic Association, vol. 13(1), pages 36-66, February.
    10. Marianne Bertrand & Dean Karlan & Sendhil Mullainathan & Eldar Shafir & Jonathan Zinman, 2010. "What's Advertising Content Worth? Evidence from a Consumer Credit Marketing Field Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(1), pages 263-306.
    11. Henry Mintzberg & James A. Waters, 1985. "Of strategies, deliberate and emergent," Strategic Management Journal, Wiley Blackwell, vol. 6(3), pages 257-272, July.
    12. Acemoglu, Daron & Johnson, Simon & Robinson, James A., 2005. "Institutions as a Fundamental Cause of Long-Run Growth," Handbook of Economic Growth, in: Philippe Aghion & Steven Durlauf (ed.), Handbook of Economic Growth, edition 1, volume 1, chapter 6, pages 385-472, Elsevier.
    13. Pritchett, Lant & Woolcock, Michael, 2004. "Solutions When the Solution is the Problem: Arraying the Disarray in Development," World Development, Elsevier, vol. 32(2), pages 191-212, February.
    14. Denizer, Cevdet & Kaufmann, Daniel & Kraay, Aart, 2013. "Good countries or good projects? Macro and micro correlates of World Bank project performance," Journal of Development Economics, Elsevier, vol. 105(C), pages 288-302.
    15. Dani Rodrik, 2007. "Introductiion to One Economics, Many Recipes: Globalization, Institutions, and Economic Growth," Introductory Chapters, in: One Economics, Many Recipes: Globalization, Institutions, and Economic Growth, Princeton University Press.
    16. Dreze, Jean & Stern, Nicholas, 1987. "The theory of cost-benefit analysis," Handbook of Public Economics, in: A. J. Auerbach & M. Feldstein (ed.), Handbook of Public Economics, edition 1, volume 2, chapter 14, pages 909-989, Elsevier.
    17. Das, Jishnu & Hammer, Jeffrey, 2007. "Money for nothing: The dire straits of medical practice in Delhi, India," Journal of Development Economics, Elsevier, vol. 83(1), pages 1-36, May.
    18. repec:unu:wpaper:wp2012-64 is not listed on IDEAS
    19. Ricardo Hausmann, 2008. "The Other Hand: High Bandwidth Development Policy," CID Working Papers 179, Center for International Development at Harvard University.
    20. Joshua Angrist & Eric Bettinger & Erik Bloom & Elizabeth King & Michael Kremer, 2002. "Vouchers for Private Schooling in Colombia: Evidence from a Randomized Natural Experiment," American Economic Review, American Economic Association, vol. 92(5), pages 1535-1558, December.
    21. Alicia H. Munnell, 1987. "Lessons from the income maintenance experiments: an overview," New England Economic Review, Federal Reserve Bank of Boston, issue May, pages 32-45.
    22. Deon Filmer & Jeffrey S. Hammer & Lant H. Pritchett, 2002. "Weak Links in the Chain II: A Prescription for Health Policy in Poor Countries," The World Bank Research Observer, World Bank, vol. 17(1), pages 47-66.
    23. Miguel Szekely, 2011. "Toward Results-Based Social Policy Design and Implementation - Working Paper 249," Working Papers 249, Center for Global Development.
    24. Banerjee, Abhijit & Duflo, Esther & , & Keniston, Daniel & Singh, Nina, 2012. "Can Institutions Be Reformed from Within? Evidence from a Randomized Experiment with the Rajasthan Police," CEPR Discussion Papers 8869, C.E.P.R. Discussion Papers.
    25. Devarajan, Shantayanan & Squire, Lyn & Suthiwart-Narueput, Sethaput, 1997. "Beyond Rate of Return: Reorienting Project Appraisal," The World Bank Research Observer, World Bank, vol. 12(1), pages 35-46, February.
    26. Rodrik, Dani, 2008. "The New Development Economics: We Shall Experiment, but How Shall We Learn?," Working Paper Series rwp08-055, Harvard University, John F. Kennedy School of Government.
    27. repec:pri:cheawb:deaton%20instruments%20of%20development%20keynes%20lecture%202009 is not listed on IDEAS
    28. Spears, Dean, 2013. "Policy Lessons from the Implementation of India’s Total Sanitation Campaign," India Policy Forum, National Council of Applied Economic Research, vol. 9(1), pages 63-104.
    29. Sanjeev Khagram & Craig Thomas & Catrina Lucero & Subarna Mathes, 2009. "Evidence for development effectiveness," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 1(3), pages 247-270.
    30. Andrews, Matt & Pritchett, Lant & Woolcock, Michael, 2013. "Escaping Capability Traps Through Problem Driven Iterative Adaptation (PDIA)," World Development, Elsevier, vol. 51(C), pages 234-244.
    31. repec:pri:cheawb:deaton%20instruments%20of%20development%20keynes%20lecture%202009.pdf is not listed on IDEAS
    32. Jessica Cohen & Pascaline Dupas, 2010. "Free Distribution or Cost-Sharing? Evidence from a Randomized Malaria Prevention Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(1), pages 1-45.
    33. Benjamin A. Olken, 2007. "Monitoring Corruption: Evidence from a Field Experiment in Indonesia," Journal of Political Economy, University of Chicago Press, vol. 115(2), pages 200-249.
    34. Felipe Barrera-Osorio & Deon Filmer, 2016. "Incentivizing Schooling for Learning: Evidence on the Impact of Alternative Targeting Approaches," Journal of Human Resources, University of Wisconsin Press, vol. 51(2), pages 461-499.
    35. Lant Pritchett, 2002. "It pays to be ignorant: A simple political economy of rigorous program evaluation," Journal of Economic Policy Reform, Taylor & Francis Journals, vol. 5(4), pages 251-269.
    36. repec:unu:wpaper:wp2012-63 is not listed on IDEAS
    37. Filmer, Deon & Hammer, Jeffrey S & Pritchett, Lant H, 2000. "Weak Links in the Chain: A Diagnosis of Health Policy in Poor Countries," The World Bank Research Observer, World Bank, vol. 15(2), pages 199-224, August.
    38. Alaka Holla & Michael Kremer, 2009. "Lessons from Randomized Evaluations in Education and Health," Working Papers 158, Center for Global Development.
    39. Joshua Angrist & Ivan Fernandez-Val, 2010. "ExtrapoLATE-ing: External Validity and Overidentification in the LATE Framework," NBER Working Papers 16566, National Bureau of Economic Research, Inc.
    40. Ostrom, Elinor, 1996. "Crossing the great divide: Coproduction, synergy, and development," World Development, Elsevier, vol. 24(6), pages 1073-1087, June.
    41. Robert Jensen, 2010. "The (Perceived) Returns to Education and the Demand for Schooling," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(2), pages 515-548.
    42. Lant Pritchett, Michael Woolcock, Matt Andrews, 2010. "Capability Traps? The Mechanisms of Persistent Implementation Failure - Working Paper 234," Working Papers 234, Center for Global Development.
    43. repec:pri:rpdevs:instruments_of_development.pdf is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Florent Bédécarrats & Isabelle Guérin & François Roubaud, 2019. "All that Glitters is not Gold. The Political Economy of Randomized Evaluations in Development," Development and Change, International Institute of Social Studies, vol. 50(3), pages 735-762, May.
    2. Hammer, Jeffrey & Spears, Dean, 2013. "Village sanitation and children's human capital : evidence from a randomized experiment by the Maharashtra government," Policy Research Working Paper Series 6580, The World Bank.
    3. Florent BEDECARRATS & Isabelle GUERIN & François ROUBAUD, 2017. "L'étalon-or des évaluations randomisées : économie politique des expérimentations aléatoires dans le domaine du développement," Working Paper 753120cd-506f-4c5f-80ed-7, Agence française de développement.
    4. repec:pri:cheawb:tscjeff2013%20paper is not listed on IDEAS
    5. Holvoet, Nathalie & Van Esbroeck, Dirk & Inberg, Liesbeth & Popelier, Lisa & Peeters, Bob & Verhofstadt, Ellen, 2018. "To evaluate or not: Evaluability study of 40 interventions of Belgian development cooperation," Evaluation and Program Planning, Elsevier, vol. 67(C), pages 189-199.
    6. Eduardo Levy Yeyati, 2019. "What Works for Active Labor Market Policies?," CID Working Papers 358, Center for International Development at Harvard University.
    7. Nadel, Sara & Pritchett, Lant, 2016. "Searching for the Devil in the Details: Learning about Development Program Design," Working Paper Series rwp16-041, Harvard University, John F. Kennedy School of Government.
    8. Cameron, Lisa & Olivia, Susan & Shah, Manisha, 2019. "Scaling up sanitation: Evidence from an RCT in Indonesia," Journal of Development Economics, Elsevier, vol. 138(C), pages 1-16.
    9. Michael Clemens, Gabriel Demombynes, 2013. "The New Transparency in Development Economics: Lessons from the Millennium Villages Controversy," Working Papers 342, Center for Global Development.
    10. Dan Levy, 2019. "Can Transparency and Accountability Programs Improve Health? Experimental Evidence from Indonesia and Tanzania," CID Working Papers 352, Center for International Development at Harvard University.
    11. Clive Bell & Lyn Squire, 2017. "Providing Policy Makers with Timely Advice: The Timeliness-Rigor Trade-off," The World Bank Economic Review, World Bank, vol. 31(2), pages 553-569.
    12. Arkedis, Jean & Creighton, Jessica & Dixit, Akshay & Fung, Archon & Kosack, Stephen & Levy, Dan & Tolmie, Courtney, 2019. "Can Transparency and Accountability Programs Improve Health? Experimental Evidence from Indonesia and Tanzania," Working Paper Series rwp19-020, Harvard University, John F. Kennedy School of Government.
    13. Ilia Murtazashvili & Jennifer Murtazashvili, 2019. "The political economy of legal titling," The Review of Austrian Economics, Springer;Society for the Development of Austrian Economics, vol. 32(3), pages 251-268, September.
    14. Lant Pritchett, Justin Sandefur, 2013. "Context Matters for Size: Why External Validity Claims and Development Practice Don't Mix-Working Paper 336," Working Papers 336, Center for Global Development.
    15. Farah Hani & Miguel Angel Santos, 2021. "Diagnosing Human Capital as a Binding Constraint to Growth: Tests, Symptoms and Prescriptions," CID Working Papers 144a, Center for International Development at Harvard University.
    16. Arkedis, Jean & Creighton, Jessica & Dixit, Akshay & Fung, Archon & Kosack, Stephen & Levy, Dan & Tolmie, Courtney, 2021. "Can transparency and accountability programs improve health? Experimental evidence from Indonesia and Tanzania," World Development, Elsevier, vol. 142(C).
    17. Woolcock, Michael, 2013. "Using Case Studies to Explore the External Validity of 'Complex' Development Interventions," Working Paper Series rwp13-048, Harvard University, John F. Kennedy School of Government.
    18. Holvoet, Nathalie & Casten, Wanda & Demissie, Eshetu Woldeyohannes & Dewachter, Sara & Gamboa, Marian Kaye C. & Adhanom, Tewelde Gebremariam & Ibrahim, Abdurahman Hamza & Makundi, Hezron & Manguni, Gr, 2023. "Theory-based evaluation of the impact of Master’s programmes in development studies: Insights from a mixed-methods and multicultural alumni action research project," Evaluation and Program Planning, Elsevier, vol. 97(C).
    19. Jorge Cornick & Alberto Trejos, 2016. "Building Public Capabilities for Productive Development Policies: Costa Rican Case Studies," IDB Publications (Working Papers) 96956, Inter-American Development Bank.
    20. Cornick, Jorge & Trejos, Alberto, 2016. "Building Public Capabilities for Productive Development Policies: Costa Rican Case Studies," IDB Publications (Working Papers) 8017, Inter-American Development Bank.
    21. Woolcock, Michael, 2013. "Using Case Studies to Explore the External Validity of 'Complex' Development Interventions," Working Paper Series rwp13-048, Harvard University, John F. Kennedy School of Government.
    22. Janssen, Matthijs J., 2019. "What bangs for your buck? Assessing the design and impact of Dutch transformative policy," Technological Forecasting and Social Change, Elsevier, vol. 138(C), pages 78-94.
    23. Matthijs Janssen, 2016. "What bangs for your bucks? Assessing the design and impact of transformative policy," Innovation Studies Utrecht (ISU) working paper series 16-05, Utrecht University, Department of Innovation Studies, revised Dec 2016.
    24. Sara Nadel and Lant Pritchett, 2016. "Searching for the Devil in the Details: Learning about Development Program Design," Working Papers 434, Center for Global Development.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lant Pritchett & Salimah Samji & Jeffrey Hammer, 2012. "It’s All About MeE: Using Structured Experiential Learning (‘e’) to Crawl the Design Space," CID Working Papers 249, Center for International Development at Harvard University.
    2. repec:pri:rpdevs:hammer_its_all_about_me is not listed on IDEAS
    3. Andrews, Matt & Pritchett, Lant & Woolcock, Michael, 2017. "Building State Capability: Evidence, Analysis, Action," OUP Catalogue, Oxford University Press, number 9780198747482.
    4. William Easterly, 2009. "Can the West Save Africa?," Journal of Economic Literature, American Economic Association, vol. 47(2), pages 373-447, June.
    5. Jörg Peters & Jörg Langbein & Gareth Roberts, 2018. "Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity," The World Bank Research Observer, World Bank, vol. 33(1), pages 34-64.
    6. Sara Nadel and Lant Pritchett, 2016. "Searching for the Devil in the Details: Learning about Development Program Design," Working Papers 434, Center for Global Development.
    7. Matt Andrews & Lant Pritchett & Michael Woolcock, 2015. "The Challenge of Building (Real) State Capability," CID Working Papers 306, Center for International Development at Harvard University.
    8. Lant Pritchett & Michael Woolcock & Matt Andrews, 2013. "Looking Like a State: Techniques of Persistent Failure in State Capability for Implementation," Journal of Development Studies, Taylor & Francis Journals, vol. 49(1), pages 1-18, January.
    9. Lant Pritchett & Michael Woolcock & Matt Andrews, 2013. "Looking Like a State: Techniques of Persistent Failure in State Capability for Implementation," Journal of Development Studies, Taylor & Francis Journals, vol. 49(1), pages 1-18, January.
    10. repec:unu:wpaper:wp2012-63 is not listed on IDEAS
    11. Larson, Greg & Ajak, Peter Biar & Pritchett, Lant, 2013. "South Sudan's Capability Trap: Building a State with Disruptive Innovation," Working Paper Series rwp13-041, Harvard University, John F. Kennedy School of Government.
    12. Karthik Muralidharan & Paul Niehaus, 2017. "Experimentation at Scale," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 103-124, Fall.
    13. Larson, Greg & Ajak, Peter Biar & Pritchett, Lant, 2013. "South Sudan's Capability Trap: Building a State with Disruptive Innovation," Working Paper Series rwp13-041, Harvard University, John F. Kennedy School of Government.
    14. Andrews, Matt & Pritchett, Lant & Woolcock, Michael, 2015. "Doing Problem Driven Work," Working Paper Series 15-073, Harvard University, John F. Kennedy School of Government.
    15. B Kelsey Jack, "undated". "Market Inefficiencies and the Adoption of Agricultural Technologies in Developing Countries," CID Working Papers 50, Center for International Development at Harvard University.
    16. Andrews, Matt & Pritchett, Lant & Woolcock, Michael, 2013. "Escaping Capability Traps Through Problem Driven Iterative Adaptation (PDIA)," World Development, Elsevier, vol. 51(C), pages 234-244.
    17. Frauke de Weijer, 2013. "A Capable State in Afghanistan: a Building Without a Foundation?," WIDER Working Paper Series wp-2013-063, World Institute for Development Economic Research (UNU-WIDER).
    18. repec:unu:wpaper:wp2012-64 is not listed on IDEAS
    19. Gisselquist, Rachel & Niño-Zarazúa, Miguel, 2013. "What can experiments tell us about how to improve governance?," MPRA Paper 49300, University Library of Munich, Germany.
    20. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2016. "Policy evaluation, randomized controlled trials, and external validity—A systematic review," Economics Letters, Elsevier, vol. 147(C), pages 51-54.
    21. Temple, Jonathan R.W., 2010. "Aid and Conditionality," Handbook of Development Economics, in: Dani Rodrik & Mark Rosenzweig (ed.), Handbook of Development Economics, edition 1, volume 5, chapter 0, pages 4415-4523, Elsevier.
    22. Gisselquist, Rachel & Niño-Zarazúa, Miguel, 2013. "What can experiments tell us about how to improve governance?," MPRA Paper 49300, University Library of Munich, Germany.
    23. Lodewijk Smets & Stephen Knack, 2018. "World Bank Policy Lending and the Quality of Public-Sector Governance," Economic Development and Cultural Change, University of Chicago Press, vol. 67(1), pages 29-54.

    More about this item

    Keywords

    development; innovation; projects; funding;
    All these keywords.

    JEL classification:

    • D61 - Microeconomics - - Welfare Economics - - - Allocative Efficiency; Cost-Benefit Analysis
    • D78 - Microeconomics - - Analysis of Collective Decision-Making - - - Positive Analysis of Policy Formulation and Implementation
    • H11 - Public Economics - - Structure and Scope of Government - - - Structure and Scope of Government
    • C40 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pri:rpdevs:hammer_its_all_about_me.pdf. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Bobray Bordelon (email available below). General contact details of provider: https://edirc.repec.org/data/rpprius.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.