Evaluating Programmes: Experiments, Non-Experiments and Propensity Scores
Evaluations of programmes ?for example, labour market interventions such as employment schemes and training courses- usually involve comparison of the performance of a treatment group (recipients of the programme) with a control group (non-recipients) as regards some response (gaining employment for example). But the idea of randomisation of individuals to groups is rarely possible in the social sciences and there may be substantial differences between groups in the distribution of individual characteristics than can affect response. Past practice in economics has been to try to use multiple regression models to adjust away the differences in observed characteristics, while also testing for sample selection bias. The Propensity Score approach, which is widely applied in epidemiology and related fields, focuses on the idea that ?matching? individuals in the groups should be compared. The appropriate matching measure is usually taken to be the prior probability of programme participation. This paper describes the key ideas of the Propensity Score method, compares it with the common approach in economics, reviews the arguments in the literature and illustrates application by reanalysis of some Irish data on training courses.
|Date of creation:||Mar 2000|
|Date of revision:|
|Contact details of provider:|| Postal: |
Phone: (353-1) 863 2000
Fax: (353-1) 863 2100
Web page: http://www.esri.ieEmail:
More information through EDIRC
References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Joel L. Horowitz & Charles F. Manski, 1996.
"Censoring of Outcomes and Regressors Due To Survey Nonresponse: Identification and Estimation Using Weights and Imputations,"
9602007, EconWPA, revised 06 Mar 1996.
- Horowitz, Joel L. & Manski, Charles F., 1998. "Censoring of outcomes and regressors due to survey nonresponse: Identification and estimation using weights and imputations," Journal of Econometrics, Elsevier, vol. 84(1), pages 37-58, May.
- Horowitz, J.L. & Manski, C.F., 1995. "Censoring of Outcomes and Regressors Due to Survey Nonresponse: Identification and estimation Using Weights and Imputations," Working Papers 95-12, University of Iowa, Department of Economics.
- Joshua D. Angrist, 1995. "Conditioning on the Probability of Selection to Control Selection Bias," NBER Technical Working Papers 0181, National Bureau of Economic Research, Inc.
- Heckman, James J & Smith, Jeffrey, 1997. "Making the Most Out of Programme Evaluations and Social Experiments: Accounting for Heterogeneity in Programme Impacts," Review of Economic Studies, Wiley Blackwell, vol. 64(4), pages 487-535, October.
- Heckman, James J & Ichimura, Hidehiko & Todd, Petra, 1998. "Matching as an Econometric Evaluation Estimator," Review of Economic Studies, Wiley Blackwell, vol. 65(2), pages 261-94, April.
- Angrist, Joshua D & Krueger, Alan B, 1991.
"Does Compulsory School Attendance Affect Schooling and Earnings?,"
The Quarterly Journal of Economics,
MIT Press, vol. 106(4), pages 979-1014, November.
- Joshua D. Angrist & Alan B. Krueger, 1990. "Does Compulsory School Attendance Affect Schooling and Earnings?," NBER Working Papers 3572, National Bureau of Economic Research, Inc.
- Joshua Angrist & Alan Krueger, 1990. "Does Compulsory School Attendance Affect Schooling and Earnings?," Working Papers 653, Princeton University, Department of Economics, Industrial Relations Section..
- James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998.
"Characterizing Selection Bias Using Experimental Data,"
NBER Working Papers
6699, National Bureau of Economic Research, Inc.
- James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," Econometrica, Econometric Society, vol. 66(5), pages 1017-1098, September.
- Imbens, Guido W & Rubin, Donald B, 1997. "Estimating Outcome Distributions for Compliers in Instrumental Variables Models," Review of Economic Studies, Wiley Blackwell, vol. 64(4), pages 555-74, October.
- J.D. Angrist & Guido W. Imbens & D.B. Rubin, 1993. "Identification of Causal Effects Using Instrumental Variables," NBER Technical Working Papers 0136, National Bureau of Economic Research, Inc.
- Heckman, James J, 1979.
"Sample Selection Bias as a Specification Error,"
Econometric Society, vol. 47(1), pages 153-61, January.
- LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-20, September.
- Rajeev H. Dehejia & Sadek Wahba, 1998. "Causal Effects in Non-Experimental Studies: Re-Evaluating the Evaluation of Training Programs," NBER Working Papers 6586, National Bureau of Economic Research, Inc.
- repec:att:wimass:9001 is not listed on IDEAS
- Heckman, J.J. & Hotz, V.J., 1988.
"Choosing Among Alternative Nonexperimental Methods For Estimating The Impact Of Social Programs: The Case Of Manpower Training,"
University of Chicago - Economics Research Center
88-12, Chicago - Economics Research Center.
- James J. Heckman, 1989. "Choosing Among Alternative Nonexperimental Methods for Estimating the Impact of Social Programs: The Case of Manpower Training," NBER Working Papers 2861, National Bureau of Economic Research, Inc.
- Jinyong Hahn, 1998. "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica, Econometric Society, vol. 66(2), pages 315-332, March.
- Little, Roderick J A, 1985. "A Note about Models for Selectivity Bias," Econometrica, Econometric Society, vol. 53(6), pages 1469-74, November.
- Newey, Whitney K & Powell, James L & Walker, James R, 1990.
"Semiparametric Estimation of Selection Models: Some Empirical Results,"
American Economic Review,
American Economic Association, vol. 80(2), pages 324-28, May.
When requesting a correction, please mention this item's handle: RePEc:esr:wpaper:wp126. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Sarah Burns)
If references are entirely missing, you can add them using this form.