IDEAS home Printed from https://ideas.repec.org/a/eso/journl/v31y2000i4p283-308.html
   My bibliography  Save this article

Evaluating State Programmes - “Natural Experiments” and Propensity Scores

Author

Listed:
  • Denis Conniffe

    (The Economic and Social Research Institute, Dublin)

  • Vanessa Gash

    (The Economic and Social Research Institute, Dublin)

  • Philip J. O'Connell

    (The Economic and Social Research Institute, Dublin)

Abstract

Evaluations of programmes — for example, labour market interventions such as employment schemes and training courses — usually involve comparison of the performance of a treatment group (recipients of the programme) with a control group (non-recipients) as regards some response (gaining employment, for example). But the ideal of randomisation of individuals to groups is rarely possible in the social sciences and there may be substantial differences between groups in the distributions of individual characteristics that can affect response. Past practice in economics has been to try to use multiple regression models to adjust away the differences in observed characteristics, while also testing for sample selection bias. The Propensity Score approach, which is widely applied in epidemiology and related fields, focuses on the idea that “matching” individuals in the groups should be compared. The appropriate matching measure is usually taken to be the prior probability of programme participation. This paper describes the key ideas of the Propensity Score method and illustrates its application by reanalysis of some Irish data on training courses.

Suggested Citation

  • Denis Conniffe & Vanessa Gash & Philip J. O'Connell, 2000. "Evaluating State Programmes - “Natural Experiments” and Propensity Scores," The Economic and Social Review, Economic and Social Studies, vol. 31(4), pages 283-308.
  • Handle: RePEc:eso:journl:v:31:y:2000:i:4:p:283-308
    as

    Download full text from publisher

    File URL: http://www.esr.ie/vol31_4/1Conniffe.pdf
    File Function: First version, 2000
    Download Restriction: no

    References listed on IDEAS

    as
    1. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 605-654.
    2. Callan, Tim & Barry Reilly, 1992. "Unions and the wage distribution in Ireland," Discussion Papers in Economics 18/92, Department of Economics, University of Sussex.
    3. James J. Heckman & Jeffrey Smith & Nancy Clements, 1997. "Making The Most Out Of Programme Evaluations and Social Experiments: Accounting For Heterogeneity in Programme Impacts," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 487-535.
    4. Heckman, J.J. & Hotz, V.J., 1988. "Choosing Among Alternative Nonexperimental Methods For Estimating The Impact Of Social Programs: The Case Of Manpower Training," University of Chicago - Economics Research Center 88-12, Chicago - Economics Research Center.
    5. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Publishing House "SINERGIA PRESS", vol. 31(3), pages 129-137.
    6. Little, Roderick J A, 1985. "A Note about Models for Selectivity Bias," Econometrica, Econometric Society, vol. 53(6), pages 1469-1474, November.
    7. Joshua D. Angrist & Alan B. Keueger, 1991. "Does Compulsory School Attendance Affect Schooling and Earnings?," The Quarterly Journal of Economics, Oxford University Press, vol. 106(4), pages 979-1014.
    8. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," Econometrica, Econometric Society, vol. 66(5), pages 1017-1098, September.
    9. Guido W. Imbens & Donald B. Rubin, 1997. "Estimating Outcome Distributions for Compliers in Instrumental Variables Models," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 555-574.
    10. Jinyong Hahn, 1998. "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica, Econometric Society, vol. 66(2), pages 315-332, March.
    11. James J. Heckman & Hidehiko Ichimura & Petra Todd, 1998. "Matching As An Econometric Evaluation Estimator," Review of Economic Studies, Oxford University Press, vol. 65(2), pages 261-294.
    12. Joshua D. Angrist, 1995. "Conditioning on the Probability of Selection to Control Selection Bias," NBER Technical Working Papers 0181, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gerry Boyle & Rory McElligott & Jim O'Leary, 2004. "Public-Private Wage Differentials in Ireland, 1994-2001," Economics, Finance and Accounting Department Working Paper Series n1421004, Department of Economics, Finance and Accounting, National University of Ireland - Maynooth.
    2. Anna Kovner & Chenyang Wei, 2012. "The private premium in public bonds," Staff Reports 553, Federal Reserve Bank of New York.
    3. Nolan, Anne, 2006. "Evaluating the Impact of Eligibility for Free Care on the Use of GP Services in Ireland: A Difference-in-Difference Matching Approach," Papers HRBWP25, Economic and Social Research Institute (ESRI).
    4. Seamus McGuinness & Philip J. O’Connell & Elish Kelly, 2013. "Carrots, No Stick, No Driver: The Employment Impact of Job Search Assistance in a Regime with Minimal Monitoring and Sanctions," Working Papers 201308, Geary Institute, University College Dublin.
    5. Reena Aggarwal & Isil Erel & René Stulz & Rohan Williamson, 2010. "Differences in Governance Practices between U.S. and Foreign Firms: Measurement, Causes, and Consequences," Review of Financial Studies, Society for Financial Studies, vol. 23(3), pages 3131-3169, March.
    6. Kelly, Elish & McGuinness, Seamus & O'Connell, Philip J., 2011. "What Can Active Labour Market Policies Do?," Papers EC1, Economic and Social Research Institute (ESRI).
    7. Halpin, Brendan & Hill, John, 2007. "Active Labour Market Programmes and Poverty Dynamics in Ireland: 1994-2001," MPRA Paper 10340, University Library of Munich, Germany.
    8. Steven Drucker & Manju Puri, 2004. "The Tying of Lending and Equity Underwriting," NBER Working Papers 10491, National Bureau of Economic Research, Inc.
    9. repec:pal:eurjdr:v:29:y:2017:i:2:d:10.1057_s41287-016-0001-3 is not listed on IDEAS
    10. McGuinness, Seamus & O'Connell, Philip J. & Kelly, Elish, 2011. "One Dummy Won't Get it: The Impact of Training Programme Type and Duration on the Employment Chances of the Unemployed in Ireland," Papers WP410, Economic and Social Research Institute (ESRI).
    11. Hutchinson, Paul & Carton, Thomas W. & Broussard, Marsha & Brown, Lisanne & Chrestman, Sarah, 2012. "Improving adolescent health through school-based health centers in post-Katrina New Orleans," Children and Youth Services Review, Elsevier, vol. 34(2), pages 360-368.
    12. Alexandre Chiavegatto Filho & Ichiro Kawachi, 2013. "Are sex-selective abortions a characteristic of every poor region? Evidence from Brazil," International Journal of Public Health, Springer;Swiss School of Public Health (SSPH+), vol. 58(3), pages 395-400, June.
    13. Jha, Anand & Cox, James, 2015. "Corporate social responsibility and social capital," Journal of Banking & Finance, Elsevier, vol. 60(C), pages 252-270.
    14. McGuinness, Seamus & O'Connell, Philip J. & Kelly, Elish & Walsh, John R., 2011. "Activation in Ireland: An Evaluation of the National Employment Action Plan," Research Series, Economic and Social Research Institute (ESRI), number RS20.
    15. McGuinness, Seamus & O'Connell, Philip J. & Kelly, Elish, 2011. "Carrots without Sticks: The Impacts of Job Search Assistance in a Regime with Minimal Monitoring and Sanctions," Papers WP409, Economic and Social Research Institute (ESRI).
    16. Aggarwal, Reena & Erel, Isil & Stulz, Rene M. & Williamson, Rohan, 2006. "Do U.S. Firms Have the Best Corporate Governance? A Cross-Country Examination of the Relation between Corporate Governance and Shareholder Wealth," Working Paper Series 2006-25, Ohio State University, Charles A. Dice Center for Research in Financial Economics.
    17. repec:eee:empfin:v:44:y:2017:i:c:p:190-208 is not listed on IDEAS
    18. Bridget Terry Long & Michal Kurlaender, 2008. "Do Community Colleges provide a Viable Pathway to a Baccalaureate Degree?," NBER Working Papers 14367, National Bureau of Economic Research, Inc.
    19. Helmreich, James E. & Pruzek, Robert M., 2009. "PSAgraphics: An R Package to Support Propensity Score Analysis," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 29(i06).
    20. Donal O'Neill, 2000. "Evaluating Labour Market Interventions," Economics, Finance and Accounting Department Working Paper Series n990300, Department of Economics, Finance and Accounting, National University of Ireland - Maynooth.
    21. Halpin, Brendan & Hill, John, 2008. "Active Labour Market Programmes and Poverty Dynamics in Ireland," MPRA Paper 10335, University Library of Munich, Germany.
    22. Hill, John & Halpin, Brendan, 2008. "The Role of Active Labour Market Programmes in Employment Policy," MPRA Paper 10498, University Library of Munich, Germany.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eso:journl:v:31:y:2000:i:4:p:283-308. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Martina Lawless). General contact details of provider: https://www.esr.ie .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.