IDEAS home Printed from https://ideas.repec.org/p/wbk/wbrwps/3877.html
   My bibliography  Save this paper

Propensity score matching and policy impact analysis - a demonstration in EViews

Author

Listed:
  • Essama-Nssah, B.

Abstract

Effective development policymaking creates a need for reliable methods of assessing effectiveness. There should be, therefore, an intimate relationship between effective policymaking and impact analysis. The goal of a development intervention defines the metric by which to assess its impact, while impact evaluation can produce reliable information on which policymakers may base decisions to modify or cancel ineffective programs and thus make the most of limited resources. This paper reviews the logic of propensity score matching (PSM) and, using data on the National Support Work Demonstration, compares that approach with other evaluation methods such as double difference, instrumental variable, and Heckman's method of selection bias correction. In addition, it demonstrates how to implement nearest-neighbor and kernel-based methods, and plot program incidence curves in E-Views. In the end, the plausibility of an evaluation method hinges critically on the correctness of the socioeconomic model underlying program design and implementation, and on the quality and quantity of available data. In any case, PSM can act as an effective adjuvant to other methods.

Suggested Citation

  • Essama-Nssah, B., 2006. "Propensity score matching and policy impact analysis - a demonstration in EViews," Policy Research Working Paper Series 3877, The World Bank.
  • Handle: RePEc:wbk:wbrwps:3877
    as

    Download full text from publisher

    File URL: http://www-wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2006/04/05/000012009_20060405111629/Rendered/PDF/wps38770rev0pdf.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Marco Caliendo & Reinhard Hujer, 2006. "The microeconometric estimation of treatment effects—An overview," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 90(1), pages 199-215, March.
    2. James J. Heckman, 1976. "The Common Structure of Statistical Models of Truncation, Sample Selection and Limited Dependent Variables and a Simple Estimator for Such Models," NBER Chapters, in: Annals of Economic and Social Measurement, Volume 5, number 4, pages 475-492, National Bureau of Economic Research, Inc.
    3. Heckman, James, 2001. "Accounting for Heterogeneity, Diversity and General Equilibrium in Evaluating Social Programmes," Economic Journal, Royal Economic Society, vol. 111(475), pages 654-699, November.
    4. Richard Blundell & Monica Costa Dias, 2009. "Alternative Approaches to Evaluation in Empirical Microeconomics," Journal of Human Resources, University of Wisconsin Press, vol. 44(3).
    5. Pedro Carneiro & Karsten T. Hansen & James J. Heckman, 2002. "Removing the Veil of Ignorance in Assessing the Distributional Impacts of Social Policies," NBER Working Papers 8840, National Bureau of Economic Research, Inc.
    6. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    7. Natalia Pimenta Monteiro, 2010. "Using propensity matching estimators to evaluate the impact of privatization on wages," Applied Economics, Taylor & Francis Journals, vol. 42(10), pages 1293-1313.
    8. Petra E. Todd & Jeffrey A. Smith, 2001. "Reconciling Conflicting Evidence on the Performance of Propensity-Score Matching Methods," American Economic Review, American Economic Association, vol. 91(2), pages 112-118, May.
    9. Marco Caliendo & Sabine Kopeinig, 2008. "Some Practical Guidance For The Implementation Of Propensity Score Matching," Journal of Economic Surveys, Wiley Blackwell, vol. 22(1), pages 31-72, February.
    10. Jalan, Jyotsna & Ravallion, Martin, 2003. "Estimating the Benefit Incidence of an Antipoverty Program by Propensity-Score Matching," Journal of Business & Economic Statistics, American Statistical Association, vol. 21(1), pages 19-30, January.
    11. Dehejia, Rajeev, 2005. "Practical propensity score matching: a reply to Smith and Todd," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 355-364.
    12. Ravallion, Martin & Chen, Shaohua, 2003. "Measuring pro-poor growth," Economics Letters, Elsevier, vol. 78(1), pages 93-99, January.
    13. B. Essama-Nssah, 2004. "Empowerment and poverty-focused evaluation," Development Southern Africa, Taylor & Francis Journals, vol. 21(3), pages 509-530.
    14. Richard Blundell & Monica Costa Dias, 2000. "Evaluation methods for non-experimental data," Fiscal Studies, Institute for Fiscal Studies, vol. 21(4), pages 427-468, January.
    15. Judy L. Baker, 2000. "Evaluating the Impact of Development Projects on Poverty : A Handbook for Practitioners," World Bank Publications - Books, The World Bank Group, number 13949, December.
    16. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    17. Ravallion, Martin, 2008. "Evaluating Anti-Poverty Programs," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 59, pages 3787-3846, Elsevier.
    18. Lokshin, Michael & Yemtsov, Ruslan, 2003. "Evaluating the impact of infrastructure rehabilitation projects on household welfare in rural Georgia," Policy Research Working Paper Series 3155, The World Bank.
    19. Barbara Sianesi, 2001. "Propensity score matching," United Kingdom Stata Users' Group Meetings 2001 12, Stata Users Group, revised 23 Aug 2001.
    20. Lee, Lung-Fei, 1983. "Generalized Econometric Models with Selectivity," Econometrica, Econometric Society, vol. 51(2), pages 507-512, March.
    21. Sascha O. Becker & Andrea Ichino, 2002. "Estimation of average treatment effects based on propensity scores," Stata Journal, StataCorp LP, vol. 2(4), pages 358-377, November.
    22. Francois Bourguignon & Luiz A. Pereira da Silva, 2003. "The Impact of Economic Policies on Poverty and Income Distribution : Evaluation Techniques and Tools," World Bank Publications - Books, The World Bank Group, number 15090, December.
    23. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    24. Smith, Jeffrey & Todd, Petra, 2005. "Rejoinder," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 365-375.
    25. Rajeev H. Dehejia & Sadek Wahba, 1998. "Causal Effects in Non-Experimental Studies: Re-Evaluating the Evaluation of Training Programs," NBER Working Papers 6586, National Bureau of Economic Research, Inc.
    26. Michael Lokshin & Zurab Sajaia, 2004. "Maximum likelihood estimation of endogenous switching regression models," Stata Journal, StataCorp LP, vol. 4(3), pages 282-289, September.
    27. Rajeev H. Dehejia & Sadek Wahba, 2002. "Propensity Score-Matching Methods For Nonexperimental Causal Studies," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 151-161, February.
    28. Robert A. Moffitt, 2004. "Introduction to the Symposium on the Econometrics of Matching," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 1-3, February.
    29. Loader, Catherine, 2004. "Smoothing: Local Regression Techniques," Papers 2004,12, Humboldt University of Berlin, Center for Applied Statistics and Economics (CASE).
    30. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
    31. Dabalen, Andrew & Paternostro, Stefano & Pierre, Gaelle, 2004. "The returns to participation in the non-farm sector in rural Rwanda," Policy Research Working Paper Series 3462, The World Bank.
    32. James J. Heckman & Hidehiko Ichimura & Petra Todd, 1998. "Matching As An Econometric Evaluation Estimator," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 65(2), pages 261-294.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Clemént, Matthieu, 2011. "Remittances and Household Expenditure Patterns in Tajikistan: A Propensity Score Matching Analysis," Asian Development Review, Asian Development Bank, vol. 28(2), pages 58-87.
    2. Seng, Kimty, 2017. "Considering the Effects of Mobile Phones on Financial Inclusion in Cambodia," MPRA Paper 82225, University Library of Munich, Germany, revised 27 Oct 2017.
    3. Wanchun Xu & Zijing Pan & Shan Lu & Liang Zhang, 2020. "Regional Heterogeneity of Application and Effect of Telemedicine in the Primary Care Centres in Rural China," IJERPH, MDPI, vol. 17(12), pages 1-15, June.
    4. Maziya, Sikhanyiso Angel, 2019. "The impact of the Europeun Union grant on access to credit and production in smallholder sugarcane agriculture in Siphofaneni, Eswatini," Research Theses 334775, Collaborative Masters Program in Agricultural and Applied Economics.
    5. Eric Stokan & Lyke Thompson & Robert J. Mahu, 2015. "Testing the Differential Effect of Business Incubators on Firm Growth," Economic Development Quarterly, , vol. 29(4), pages 317-327, November.
    6. Himaz, Rozana, 2008. "Welfare Grants and Their Impact on Child Health: The Case of Sri Lanka," World Development, Elsevier, vol. 36(10), pages 1843-1857, October.
    7. Himaz, Rozana, 2009. "The impact of parental death on schooling and subjective wellbeing: Evidence from Ethiopia using longitudinal data," MPRA Paper 21735, University Library of Munich, Germany.
    8. Ziqi Wang & Ziyao Huang & Jingjing Cai, 2023. "Does the One-Child Policy Improve Chinese Human Capital? A Propensity Score Matching Analysis," Sustainability, MDPI, vol. 15(16), pages 1-24, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    2. Ravallion, Martin, 2008. "Evaluating Anti-Poverty Programs," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 59, pages 3787-3846, Elsevier.
    3. Jochen Kluve & Boris Augurzky, 2007. "Assessing the performance of matching algorithms when selection into treatment is strong," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(3), pages 533-557.
    4. Richard Blundell & Monica Costa Dias, 2009. "Alternative Approaches to Evaluation in Empirical Microeconomics," Journal of Human Resources, University of Wisconsin Press, vol. 44(3).
    5. Miguel Angel Malo & Fernando Muñoz-Bullón, 2006. "Employment promotion measures and the quality of the job match for persons with disabilities," Hacienda Pública Española / Review of Public Economics, IEF, vol. 179(4), pages 79-111, September.
    6. Jones A.M & Rice N, 2009. "Econometric Evaluation of Health Policies," Health, Econometrics and Data Group (HEDG) Working Papers 09/09, HEDG, c/o Department of Economics, University of York.
    7. Kluve, Jochen & Lehmann, Hartmut & Schmidt, Christoph M., 2008. "Disentangling Treatment Effects of Active Labor Market Policies: The Role of Labor Force Status Sequences," Labour Economics, Elsevier, vol. 15(6), pages 1270-1295, December.
    8. Marco Caliendo & Sabine Kopeinig, 2008. "Some Practical Guidance For The Implementation Of Propensity Score Matching," Journal of Economic Surveys, Wiley Blackwell, vol. 22(1), pages 31-72, February.
    9. Tommaso Nannicini, 2007. "Simulation-based sensitivity analysis for matching estimators," Stata Journal, StataCorp LP, vol. 7(3), pages 334-350, September.
    10. Ayala, Luis & Rodríguez, Magdalena, 2013. "Health-related effects of welfare-to-work policies," Social Science & Medicine, Elsevier, vol. 93(C), pages 103-112.
    11. Edgar Silva Quintero & José Alberto Molina & J. Ignacio Gimenez-Nadal, 2016. "How Forced Displacements Caused by a Violent Conflict Affect Wages in Colombia," Working Papers id:10876, eSocialSciences.
    12. Jose C. Galdo & Jeffrey Smith & Dan Black, 2008. "Bandwidth Selection and the Estimation of Treatment Effects with Unbalanced Data," Annals of Economics and Statistics, GENES, issue 91-92, pages 189-216.
    13. Alm, Bastian & Bade, Franz-Josef, 2009. "The impact of firm subsidies: Evaluating German regional policy," EconStor Preprints 103402, ZBW - Leibniz Information Centre for Economics.
    14. Ashimwe, Olive, 2016. "An Economic Analysis Of Impact Of Weather Index-Based Crop Insurance On Household Income In Huye District Of Rwanda," Research Theses 276460, Collaborative Masters Program in Agricultural and Applied Economics.
    15. Huber, Martin & Lechner, Michael & Wunsch, Conny, 2010. "How to Control for Many Covariates? Reliable Estimators Based on the Propensity Score," IZA Discussion Papers 5268, Institute of Labor Economics (IZA).
    16. Ashimwe, Olive, 2016. "An Economic Analysis Of Impact Of Weather Index-Based Crop Insurance On Household Income In Huye District Of Rwanda," Research Theses 265675, Collaborative Masters Program in Agricultural and Applied Economics.
    17. Huber, Martin & Lechner, Michael & Wunsch, Conny, 2013. "The performance of estimators based on the propensity score," Journal of Econometrics, Elsevier, vol. 175(1), pages 1-21.
    18. Antonio Trujillo & Jorge Portillo & John Vernon, 2005. "The Impact of Subsidized Health Insurance for the Poor: Evaluating the Colombian Experience Using Propensity Score Matching," International Journal of Health Economics and Management, Springer, vol. 5(3), pages 211-239, September.
    19. Eliasson, Kent, 2006. "How Robust is the Evidence on the Returns to College Choice? Results Using Swedish Administrative Data," Umeå Economic Studies 692, Umeå University, Department of Economics.
    20. Peter R. Mueser & Kenneth R. Troske & Alexey Gorislavsky, 2007. "Using State Administrative Data to Measure Program Performance," The Review of Economics and Statistics, MIT Press, vol. 89(4), pages 761-783, November.

    More about this item

    Keywords

    Poverty Monitoring&Analysis; Poverty Impact Evaluation; Statistical&Mathematical Sciences; Scientific Research&Science Parks; Science Education;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wbk:wbrwps:3877. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Roula I. Yazigi (email available below). General contact details of provider: https://edirc.repec.org/data/dvewbus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.