Using Performance Standards to Evaluate Social Programs with Incomplete Outcome Data: General Issues and Application to a Higher Education Block Grant Program
AbstractThe basic idea of program evaluation is both simple and appealing. Program outcomes are measured and compared to some minimum performance standard or threshold. In practice, however, evaluation is quite difficult. Two fundamental problems of outcome measurement must be addressed. The first, which we call the problem of auxiliary outcomes, is that we do not observe outcome of interest. The second, which we call the problem of counterfactual outcomes, is that we do not observe the threshold standard. This paper examines how performance standards should be set and applied in the face of these problems in measuring outcomes. In particular, we consider the problem of evaluating the new World Bank-sponsored Quality of Under-graduate Education (QUE) program. This competitive block grant program is to be judged by the program's effects on student outcomes, not by the particular ways in which grantee depart-ments use their funds. Our central message is that the proper way to implement standards varies with the prior information that the evaluator can credibly bring to bear to compensate for incomplete outcome data. An evaluator, confronted with the auxiliary and counter-factual outcomes problems, should combine the available data with credible assumptions on treatment and outcomes. Given this information, the performance of a program may be deemed acceptable, unacceptable, or indeterminate.
Download InfoTo our knowledge, this item is not available for download. To find whether it is available, there are three options:
1. Check below under "Related research" whether another version of this item is available online.
2. Check on the provider's web page whether it is in fact available.
3. Perform a search for a similarly titled item that would be available.
Bibliographic InfoPaper provided by Institute for Policy Resarch at Northwestern University in its series IPR working papers with number 00-1.
Date of creation:
Date of revision:
Other versions of this item:
- John V. Pepper, 1998. "Using Performance Standards to Evaluate Social Programs with Incomplete Outcome Data: General Issues and Application to a Higher Education Block Grant Program," Virginia Economics Online Papers 312, University of Virginia, Department of Economics.
- C40 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - General
- H43 - Public Economics - - Publicly Provided Goods - - - Project Evaluation; Social Discount Rate
- I22 - Health, Education, and Welfare - - Education - - - Educational Finance; Financial Aid
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- John V. Pepper, 1999.
"What Do Welfare-to-Work Demonstrations Reveal to Welfare Reformers?,"
JCPR Working Papers
105, Northwestern University/University of Chicago Joint Center for Poverty Research.
- John V. Pepper, 1999. "What Do Welfare-to-Work Demonstrations Reveal to Welfare Reformers?," Virginia Economics Online Papers 317, University of Virginia, Department of Economics.
- Manski, Charles F, 1990.
"Nonparametric Bounds on Treatment Effects,"
American Economic Review,
American Economic Association, vol. 80(2), pages 319-23, May.
- Charles F. Manski, 1997.
"Monotone Treatment Response,"
Econometric Society, vol. 65(6), pages 1311-1334, November.
- Heckman, James J & Smith, Jeffrey, 1997. "Making the Most Out of Programme Evaluations and Social Experiments: Accounting for Heterogeneity in Programme Impacts," Review of Economic Studies, Wiley Blackwell, vol. 64(4), pages 487-535, October.
- Manski, C.F., 1990. "The Selection Problem," Working papers 90-12, Wisconsin Madison - Social Systems.
- repec:att:wimass:9217 is not listed on IDEAS
- Heckman, James J. & Robb, Richard Jr., 1985. "Alternative methods for evaluating the impact of interventions : An overview," Journal of Econometrics, Elsevier, vol. 30(1-2), pages 239-267.
- Bjorklund, Anders & Moffitt, Robert, 1987. "The Estimation of Wage Gains and Welfare Gains in Self-selection," The Review of Economics and Statistics, MIT Press, vol. 69(1), pages 42-49, February.
- Charles F. Manski & John V. Pepper, 1998.
"Monotone Instrumental Variables: With an Application to the Returns to Schooling,"
Virginia Economics Online Papers
308, University of Virginia, Department of Economics.
- Charles F. Manski & John V. Pepper, 2000. "Monotone Instrumental Variables, with an Application to the Returns to Schooling," Econometrica, Econometric Society, vol. 68(4), pages 997-1012, July.
- Charles F. Manski & John V. Pepper, 1998. "Monotone Instrumental Variables with an Application to the Returns to Schooling," NBER Technical Working Papers 0224, National Bureau of Economic Research, Inc.
- John V. Pepper, 2000. "The Intergenerational Transmission Of Welfare Receipt: A Nonparametric Bounds Analysis," The Review of Economics and Statistics, MIT Press, vol. 82(3), pages 472-488, August.
- Daniel Friedlander & David H. Greenberg & Philip K. Robins, 1997. "Evaluating Government Training Programs for the Economically Disadvantaged," Journal of Economic Literature, American Economic Association, vol. 35(4), pages 1809-1855, December.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Thomas Krichel).
If references are entirely missing, you can add them using this form.