IDEAS home Printed from https://ideas.repec.org/p/vir/virpap/312.html
   My bibliography  Save this paper

Using Performance Standards to Evaluate Social Programs with Incomplete Outcome Data: General Issues and Application to a Higher Education Block Grant Program

Author

Listed:
  • John V. Pepper

    ()

Abstract

The basic idea of program evaluation is both simple and appealing. Program outcomes are measured and compared to some minimum performance standard or threshold. In practice, however, evaluation is quite difficult. Two fundamental problems of outcome measurement must be addressed. The first, which we call the problem of auxiliary outcomes, is that we do not observe outcome of interest. The second, which we call the problem of counterfactual outcomes, is that we do not observe the threshold standard. This paper examines how performance standards should be set and applied in the face of these problems in measuring outcomes. In particular, we consider the problem of evaluating the new World Bank sponsored Quality of Undergraduate Education (QUE) program. This competitive block grant program is to be judged by the program's effects on student outcomes, not by the particular ways in which the grantee departments use their funds. Our central message is that the proper way to implement standards varies with the prior information that the evaluator can credibly bring to bear to compensate for incomplete outcome data. An evaluator, confronted with the auxiliary and counterfactual outcomes problems, should combine the available data with credible assumptions on treatments and outcomes. Given this information, the performance of a program may be deemed acceptable, unacceptable or indeterminate.

Suggested Citation

  • John V. Pepper, 1998. "Using Performance Standards to Evaluate Social Programs with Incomplete Outcome Data: General Issues and Application to a Higher Education Block Grant Program," Virginia Economics Online Papers 312, University of Virginia, Department of Economics.
  • Handle: RePEc:vir:virpap:312
    as

    Download full text from publisher

    File URL: http://www.virginia.edu/economics/RePEc/vir/virpap/papers/virpap312.pdf
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. John V. Pepper, 1999. "What Do Welfare-to-Work Demonstrations Reveal to Welfare Reformers?," JCPR Working Papers 105, Northwestern University/University of Chicago Joint Center for Poverty Research.
    2. Manski, Charles F, 1990. "Nonparametric Bounds on Treatment Effects," American Economic Review, American Economic Association, vol. 80(2), pages 319-323, May.
    3. Charles F. Manski, 1997. "Monotone Treatment Response," Econometrica, Econometric Society, vol. 65(6), pages 1311-1334, November.
    4. Manski, C.F., 1992. "Identification Problems in the Social Sciences," Working papers 9217, Wisconsin Madison - Social Systems.
    5. James J. Heckman & Jeffrey Smith & Nancy Clements, 1997. "Making The Most Out Of Programme Evaluations and Social Experiments: Accounting For Heterogeneity in Programme Impacts," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 487-535.
    6. Manski, C.F., 1990. "The Selection Problem," Working papers 90-12, Wisconsin Madison - Social Systems.
    7. Charles F. Manski & John V. Pepper, 2000. "Monotone Instrumental Variables, with an Application to the Returns to Schooling," Econometrica, Econometric Society, vol. 68(4), pages 997-1012, July.
    8. John V. Pepper, 2000. "The Intergenerational Transmission Of Welfare Receipt: A Nonparametric Bounds Analysis," The Review of Economics and Statistics, MIT Press, vol. 82(3), pages 472-488, August.
    9. Daniel Friedlander & David H. Greenberg & Philip K. Robins, 1997. "Evaluating Government Training Programs for the Economically Disadvantaged," Journal of Economic Literature, American Economic Association, vol. 35(4), pages 1809-1855, December.
    10. V. Joseph Hotz & Charles H. Mullin & Seth G. Sanders, 1997. "Bounding Causal Effects Using Data from a Contaminated Natural Experiment: Analysing the Effects of Teenage Childbearing," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 575-603.
    11. Heckman, James J. & Robb, Richard Jr., 1985. "Alternative methods for evaluating the impact of interventions : An overview," Journal of Econometrics, Elsevier, vol. 30(1-2), pages 239-267.
    12. Bjorklund, Anders & Moffitt, Robert, 1987. "The Estimation of Wage Gains and Welfare Gains in Self-selection," The Review of Economics and Statistics, MIT Press, vol. 69(1), pages 42-49, February.
    Full references (including those not matched with items on IDEAS)

    More about this item

    JEL classification:

    • C40 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - General
    • H43 - Public Economics - - Publicly Provided Goods - - - Project Evaluation; Social Discount Rate
    • I22 - Health, Education, and Welfare - - Education - - - Educational Finance; Financial Aid

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:vir:virpap:312. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Debby Stanford). General contact details of provider: http://www.virginia.edu/economics/home.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.