Advanced Search
MyIDEAS: Login

A Comparison of Model-based and Design-based Impact Evaluations of Interventions in Developing Countries

Contents:

Author Info

  • Henrik Hansen

    ()
    (Institute of Food and Resource Economics, University of Copenhagen)

  • Ninja Ritter Klejnstrup

    (Evaluation Department, Ministry of Foreign Affairs of Denmark, Danida)

  • Ole Winckler Andersen

    (Evaluation Department, Ministry of Foreign Affairs of Denmark, Danida)

Abstract

We argue that non-experimental impact estimators will continue to be needed for evaluations of interventions in developing countries as social experiments, for various reasons, will never be the most preferred approach. In a survey of four studies that empirically compare the performance of experimental and non-experimental impact estimates using data from development interventions, we show that the preferred non-experimental estimators are unbiased. We try to explain the reasons why the non-experimental estimators perform better in the context of development interventions than American job-market interventions. We also use the survey as a source for suggestions for implementation and assessment of non-experimental impact evaluations. Our main suggestion is to be more careful and precise in the formulation of the statistical model for the assignment into the program and also to use the assignment information for model-based systematic sampling.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://okonomi.foi.dk/workingpapers/WPpdf/WP2011/WP_2011_16_model_vs_design.pdf
Download Restriction: no

Bibliographic Info

Paper provided by University of Copenhagen, Department of Food and Resource Economics in its series IFRO Working Paper with number 2011/16.

as in new window
Length: 25 pages
Date of creation: Dec 2011
Date of revision:
Handle: RePEc:foi:wpaper:2011_16

Contact details of provider:
Email:
Web page: http://www.ifro.ku.dk/
More information through EDIRC

Related research

Keywords: Development; impact; non-experimental; social experiment; within-study;

Find related papers by JEL classification:

This paper has been announced in the following NEP Reports:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. McKenzie, David & Gibson, John & Stillman, Steven, 2006. "How Important Is Selection? Experimental vs. Non-Experimental Measures of the Income Gains from Migration," IZA Discussion Papers 2087, Institute for the Study of Labor (IZA).
  2. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-55, June.
  3. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," Econometrica, Econometric Society, vol. 66(5), pages 1017-1098, September.
  4. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
  5. Paul Glewwe & Michael Kremer & Sylvie Moulin & Eric Zitzewitz, 2000. "Retrospective vs. Prospective Analyses of School Inputs: The Case of Flip Charts in Kenya," NBER Working Papers 8018, National Bureau of Economic Research, Inc.
  6. Juan Jose Diaz & Sudhanshu Handa, 2006. "An Assessment of Propensity Score Matching as a Nonexperimental Impact Estimator: Evidence from Mexico’s PROGRESA Program," Journal of Human Resources, University of Wisconsin Press, vol. 41(2).
  7. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Publishing House "SINERGIA PRESS", vol. 31(3), pages 129-137.
  8. Elster, Jon, 1989. "Social Norms and Economic Theory," Journal of Economic Perspectives, American Economic Association, vol. 3(4), pages 99-117, Fall.
  9. Christopher B. Barrett & Michael R. Carter, 2010. "The Power and Pitfalls of Experiments in Development Economics: Some Non-random Reflections," Applied Economic Perspectives and Policy, Agricultural and Applied Economics Association, vol. 32(4), pages 515-548.
  10. Guido M. Imbens & Jeffrey M. Wooldridge, 2008. "Recent Developments in the Econometrics of Program Evaluation," NBER Working Papers 14251, National Bureau of Economic Research, Inc.
  11. Stutzer, Alois & Lalive, Rafael, 2001. "The Role of Social Work Norms in Job Searching and Subjective Well-Being," IZA Discussion Papers 300, Institute for the Study of Labor (IZA).
  12. J. A. Hausman, 1976. "Specification Tests in Econometrics," Working papers 185, Massachusetts Institute of Technology (MIT), Department of Economics.
  13. Sudhanshu Handa & John Maluccio, 2008. "Matching the gold standard: Comparing experimental and non-experimental evaluation techniques for a geographically targeted program," Middlebury College Working Paper Series 0813, Middlebury College, Department of Economics.
  14. Dehejia, Rajeev, 2005. "Practical propensity score matching: a reply to Smith and Todd," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 355-364.
  15. Heckman, James J. & Robb, Richard Jr., 1985. "Alternative methods for evaluating the impact of interventions : An overview," Journal of Econometrics, Elsevier, vol. 30(1-2), pages 239-267.
  16. Rajeev H. Dehejia & Sadek Wahba, 1998. "Causal Effects in Non-Experimental Studies: Re-Evaluating the Evaluation of Training Programs," NBER Working Papers 6586, National Bureau of Economic Research, Inc.
  17. Thomas D. Cook & William R. Shadish & Vivian C. Wong, 2008. "Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within-study comparisons," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 27(4), pages 724-750.
  18. James J. Heckman & Vytlacil, Edward J., 2007. "Econometric Evaluation of Social Programs, Part I: Causal Models, Structural Models and Econometric Policy Evaluation," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 70 Elsevier.
  19. Andrew Clark, 2001. "Unemployment As A Social Norm: Psychological Evidence from Panel Data," DELTA Working Papers 2001-17, DELTA (Ecole normale supérieure).
  20. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-20, September.
  21. Paul J. Gertler & Sebastian Martinez & Patrick Premand & Laura B. Rawlings & Christel M. J. Vermeersch, 2011. "Impact Evaluation in Practice," World Bank Publications, The World Bank, number 2550, March.
  22. Thomas Fraker & Rebecca Maynard, 1987. "The Adequacy of Comparison Group Designs for Evaluations of Employment-Related Programs," Journal of Human Resources, University of Wisconsin Press, vol. 22(2), pages 194-227.
  23. David H. Greenberg & Charles Michalopoulos & Philip K. Robin, 2006. "Do experimental and nonexperimental evaluations give different answers about the effectiveness of government-funded training programs?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 25(3), pages 523-552.
Full references (including those not matched with items on IDEAS)

Citations

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:foi:wpaper:2011_16. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Geir Tveit).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.