IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp827.html
   My bibliography  Save this paper

An Evaluation of the Performance of Regression Discontinuity Design on PROGRESA

Author

Listed:
  • Buddelmeyer, Hielke

    () (Melbourne Institute of Applied Economic and Social Research)

  • Skoufias, Emmanuel

    () (World Bank)

Abstract

While providing the most reliable method of evaluating social programs, randomized experiments in developing and developed countries alike are accompanied by political risks and ethical issues that jeopardize the chances of adopting them. In this paper we use a unique data set from rural Mexico collected for the purposes of evaluating the impact of the PROGRESA poverty alleviation program to examine the performance of a quasiexperimental estimator, the Regression Discontinuity Design (RDD). Using as a benchmark the impact estimates based on the experimental nature of the sample, we examine how estimates differ when we use the RD design as the estimator for evaluating program impact on two key indicators: child school attendance and child work. Overall the performance of the RDD performance was remarkably good. The RDD estimates of program impact agreed with the experimental estimates in 10 out of the 12 possible cases. The two cases in which the RDD method failed to reveal any significant program impact on the school attendance of boys and girls were in the first year of the program (round 3). RDD estimates comparable to the experimental estimates were obtained when we used as a comparison group children from non-eligible households in the control localities.

Suggested Citation

  • Buddelmeyer, Hielke & Skoufias, Emmanuel, 2003. "An Evaluation of the Performance of Regression Discontinuity Design on PROGRESA," IZA Discussion Papers 827, Institute for the Study of Labor (IZA).
  • Handle: RePEc:iza:izadps:dp827
    as

    Download full text from publisher

    File URL: http://ftp.iza.org/dp827.pdf
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Emmanuel Skoufias & Susan Wendy Parker, 2001. "Conditional Cash Transfers and Their Impact on Child Work and Schooling: Evidence from the PROGRESA Program in Mexico," ECONOMIA JOURNAL, THE LATIN AMERICAN AND CARIBBEAN ECONOMIC ASSOCIATION - LACEA, vol. 0(Fall 2001), pages 45-96, August.
    2. Schultz, T. Paul, 2001. "School subsidies for the poor," FCND briefs 102, International Food Policy Research Institute (IFPRI).
    3. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 605-654.
    4. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    5. Petra E. Todd & Jeffrey A. Smith, 2001. "Reconciling Conflicting Evidence on the Performance of Propensity-Score Matching Methods," American Economic Review, American Economic Association, vol. 91(2), pages 112-118, May.
    6. Rajeev H. Dehejia & Sadek Wahba, 2002. "Propensity Score-Matching Methods For Nonexperimental Causal Studies," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 151-161, February.
    7. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics,in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097 Elsevier.
    8. Wilbert van der Klaauw, 2002. "Estimating the Effect of Financial Aid Offers on College Enrollment: A Regression-Discontinuity Approach," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 43(4), pages 1249-1287, November.
    9. Judy L. Baker, 2000. "Evaluating the Impact of Development Projects on Poverty : A Handbook for Practitioners," World Bank Publications, The World Bank, number 13949.
    10. Jinyong Hahn & Petra Todd & Wilbert Van der Klaauw, 1999. "Evaluating the Effect of an Antidiscrimination Law Using a Regression-Discontinuity Design," NBER Working Papers 7131, National Bureau of Economic Research, Inc.
    11. Paul Schultz, T., 2004. "School subsidies for the poor: evaluating the Mexican Progresa poverty program," Journal of Development Economics, Elsevier, vol. 74(1), pages 199-250, June.
    12. Behrman, Jere R. & Deolalikar, Anil B., 1988. "Health and nutrition," Handbook of Development Economics,in: Hollis Chenery & T.N. Srinivasan (ed.), Handbook of Development Economics, edition 1, volume 1, chapter 14, pages 631-711 Elsevier.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Galiani, Sebastian & McEwan, Patrick J., 2013. "The heterogeneous impact of conditional cash transfers," Journal of Public Economics, Elsevier, vol. 103(C), pages 85-96.
    2. Frandsen, Brigham R. & Frölich, Markus & Melly, Blaise, 2012. "Quantile treatment effects in the regression discontinuity design," Journal of Econometrics, Elsevier, vol. 168(2), pages 382-395.
    3. Viviane Azevedo & César P. Bouillon & Patricia Yáñez-Pagans, 2009. "How Much Are We Willing To Pay to Send Poor Adolescents to School?: Simulating Changes to Mexico`s Oportunidades in Urban Areas," IDB Publications (Working Papers) 1655, Inter-American Development Bank.
    4. Rafael Lalive & M. Alejandra Cattaneo, 2009. "Social Interactions and Schooling Decisions," The Review of Economics and Statistics, MIT Press, vol. 91(3), pages 457-477, August.
    5. Cook, Thomas D., 2008. ""Waiting for Life to Arrive": A history of the regression-discontinuity design in Psychology, Statistics and Economics," Journal of Econometrics, Elsevier, vol. 142(2), pages 636-654, February.
    6. Battistin, Erich & Rettore, Enrico, 2008. "Ineligibles and eligible non-participants as a double comparison group in regression-discontinuity designs," Journal of Econometrics, Elsevier, vol. 142(2), pages 715-730, February.
    7. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, Elsevier.
    8. Philip M. Gleason & Alexandra M. Resch & Jillian A. Berk, 2012. "Replicating Experimental Impact Estimates Using a Regression Discontinuity Approach," Mathematica Policy Research Reports c7ecf8b35c8d402491d09b2d0, Mathematica Policy Research.
    9. Maureen A. Pirog & Anne L. Buffardi & Colleen K. Chrisinger & Pradeep Singh & John Briney, 2009. "Are the alternatives to randomized assignment nearly as good? Statistical corrections to nonrandomized evaluations," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 28(1), pages 169-172.
    10. Mora Rivera, José Jorge. & Arellano-González, Jesús., 2012. "Migration and Remittances Effects on Consumption of the Poorest : the Mexican Case," Panorama Económico, Escuela Superior de Economía, Instituto Politécnico Nacional, vol. 0(14), pages 121-163, primer se.
    11. Burgert, Derik, 2005. "Schwellenwerte im deutschen Kündigungsschutzrecht Ein Beschäftigungshindernis für kleine Unternehmen?," MPRA Paper 5969, University Library of Munich, Germany.
    12. David S. Lee & Thomas Lemieux, 2010. "Regression Discontinuity Designs in Economics," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 281-355, June.
    13. Derik Burgert, 2005. "Schwellenwerte im deutschen Kündigungsschutzrecht - Ein Beschäftigungshindernis für kleine Unternehmen?," FFB-Discussionpaper 51, Research Institute on Professions (Forschungsinstitut Freie Berufe (FFB)), LEUPHANA University Lüneburg.
    14. Peralta, Maria Alexandra & Swinton, Scott M. & Maredia, Mywish K., 2011. "Accounting for selection bias in impact analysis of a rural development program: An application using propensity score matching," 2012 Conference, August 18-24, 2012, Foz do Iguacu, Brazil 126398, International Association of Agricultural Economists.
    15. Burgert, Derik, 2006. "Einstellungschancen von Älteren – Wie wirkt der Schwellenwert im Kündigungsschutz?," MPRA Paper 5846, University Library of Munich, Germany.
    16. Eliana Cardoso & Andre Portela Souza, 2004. "The Impact of Cash Transfers on Child Labor and School Attendance in Brazil," Vanderbilt University Department of Economics Working Papers 0407, Vanderbilt University Department of Economics.
    17. Nelson, Suzanne & Frakenberger, Tim & Brown, Vicky & Presnall, Carrie & Downen, Jeanne, 2015. "Ex-Post impact assessment review of IFPRI’s research program on social protection, 2000–2012," Impact assessments 40, International Food Policy Research Institute (IFPRI).
    18. Solomon Asfaw & Silvio Daidone & Benjamin Davis & Josh Dewbre & Alessandro Romeo & Paul Winters & Katia Covarrubias & Habiba Djebbari, 2012. "Analytical Framework for Evaluating the Productive Impact of Cash Transfer Programmes on Household Behaviour ? Methodological Guidelines for the From Protection to Production Project," Working Papers 101, International Policy Centre for Inclusive Growth.
    19. Nirav Mehta, 2015. "An Economic Approach to Generalize Findings from Regression-Discontinuity Designs," University of Western Ontario, Centre for Human Capital and Productivity (CHCP) Working Papers 20156, University of Western Ontario, Centre for Human Capital and Productivity (CHCP).
    20. Jere R. Behrman & Susan W. Parker & Petra E. Todd, 2005. "Long-Term Impacts of the Oportunidades Conditional Cash Transfer Program on Rural Youth in Mexico," Ibero America Institute for Econ. Research (IAI) Discussion Papers 122, Ibero-America Institute for Economic Research.
    21. Nguyen Viet, Cuong, 2006. "An Introduction to Alternative Methods in Program Impact Evaluation," MPRA Paper 24900, University Library of Munich, Germany.
    22. Frenette, Marc, 2011. "Is Debt Relief as Good as Liquidity? The Impact of Prospective Student Debt on Post-Secondary Attendance among Low-Income Youth," CLSSRN working papers clsrn_admin-2011-7, Vancouver School of Economics, revised 29 Mar 2011.
    23. Burgert, Derik, 2005. "The Impact of German Job Protection Legislation on Job Creation in Small Establishments - An Application of the Regression Discontinuity Design," MPRA Paper 5971, University Library of Munich, Germany.

    More about this item

    Keywords

    Mexico; PROGRESA; Regression Discontinuity; treatment effects;

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy
    • I32 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - Measurement and Analysis of Poverty
    • J13 - Labor and Demographic Economics - - Demographic Economics - - - Fertility; Family Planning; Child Care; Children; Youth

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp827. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Mark Fallak). General contact details of provider: http://www.iza.org .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.