Using Performance Standards to Evaluate Social Programs with Incomplete Outcome Data: General Issues and Application to a Higher Education Block Grant Program
The basic idea of program evaluation is both simple and appealing. Program outcomes are measured and compared to some minimum performance standard or threshold. In practice, however, evaluation is quite difficult. Two fundamental problems of outcome measurement must be addressed. The first, which we call the problem of auxiliary outcomes, is that we do not observe outcome of interest. The second, which we call the problem of counterfactual outcomes, is that we do not observe the threshold standard. This paper examines how performance standards should be set and applied in the face of these problems in measuring outcomes. In particular, we consider the problem of evaluating the new World Bank-sponsored Quality of Under-graduate Education (QUE) program. This competitive block grant program is to be judged by the program's effects on student outcomes, not by the particular ways in which grantee depart-ments use their funds. Our central message is that the proper way to implement standards varies with the prior information that the evaluator can credibly bring to bear to compensate for incomplete outcome data. An evaluator, confronted with the auxiliary and counter-factual outcomes problems, should combine the available data with credible assumptions on treatment and outcomes. Given this information, the performance of a program may be deemed acceptable, unacceptable, or indeterminate.
To our knowledge, this item is not available for
download. To find whether it is available, there are three
1. Check below under "Related research" whether another version of this item is available online.
2. Check on the provider's web page whether it is in fact available.
3. Perform a search for a similarly titled item that would be available.
|Date of creation:|
|Contact details of provider:|| Postal: 2040 Sheridan Road, Evanston, IL 60208-4100|
Web page: http://www.nwu.edu/IPR/publications/wpindex1.html
More information through EDIRC
References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- V. Joseph Hotz & Charles H. Mullin & Seth G. Sanders, 1997. "Bounding Causal Effects Using Data from a Contaminated Natural Experiment: Analysing the Effects of Teenage Childbearing," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 575-603.
- John V. Pepper, 2000. "The Intergenerational Transmission Of Welfare Receipt: A Nonparametric Bounds Analysis," The Review of Economics and Statistics, MIT Press, vol. 82(3), pages 472-488, August.
- Heckman, James J. & Robb, Richard Jr., 1985. "Alternative methods for evaluating the impact of interventions : An overview," Journal of Econometrics, Elsevier, vol. 30(1-2), pages 239-267.
- Manski, C.F., 1996.
"Monotone Treatment Response,"
9604, Wisconsin Madison - Social Systems.
- Manski, C.F., 1990. "The Selection Problem," Working papers 90-12, Wisconsin Madison - Social Systems.
- John V. Pepper, 1999.
"What Do Welfare-to-Work Demonstrations Reveal to Welfare Reformers?,"
Virginia Economics Online Papers
317, University of Virginia, Department of Economics.
- John V. Pepper, 1999. "What Do Welfare-to-Work Demonstrations Reveal to Welfare Reformers?," JCPR Working Papers 105, Northwestern University/University of Chicago Joint Center for Poverty Research.
- Daniel Friedlander & David H. Greenberg & Philip K. Robins, 1997. "Evaluating Government Training Programs for the Economically Disadvantaged," Journal of Economic Literature, American Economic Association, vol. 35(4), pages 1809-1855, December.
- Manski, Charles F, 1990.
"Nonparametric Bounds on Treatment Effects,"
American Economic Review,
American Economic Association, vol. 80(2), pages 319-323, May.
- Charles F. Manski & John V. Pepper, 2000.
"Monotone Instrumental Variables, with an Application to the Returns to Schooling,"
Econometric Society, vol. 68(4), pages 997-1012, July.
- Charles F. Manski & John V. Pepper, 1998. "Monotone Instrumental Variables: With an Application to the Returns to Schooling," Virginia Economics Online Papers 308, University of Virginia, Department of Economics.
- Charles F. Manski & John V. Pepper, 1998. "Monotone Instrumental Variables with an Application to the Returns to Schooling," NBER Technical Working Papers 0224, National Bureau of Economic Research, Inc.
- Bjorklund, Anders & Moffitt, Robert, 1987. "The Estimation of Wage Gains and Welfare Gains in Self-selection," The Review of Economics and Statistics, MIT Press, vol. 69(1), pages 42-49, February.
- Manski, C.F., 1992. "Identification Problems in the Social Sciences," Working papers 9217, Wisconsin Madison - Social Systems.
- James J. Heckman & Jeffrey Smith & Nancy Clements, 1997. "Making The Most Out Of Programme Evaluations and Social Experiments: Accounting For Heterogeneity in Programme Impacts," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 487-535.
When requesting a correction, please mention this item's handle: RePEc:wop:nwuipr:00-1. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Thomas Krichel)
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If references are entirely missing, you can add them using this form.
If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.
Please note that corrections may take a couple of weeks to filter through the various RePEc services.