Advanced Search
MyIDEAS: Login to save this paper or follow this series

Public Policy Evaluation: Introduction to Quantitative Methodologies


Author Info

  • Takis Venetoklis


This paper is a survey which describes and explains in non-technical terms the logic behind various methodologies used in conducting retrospective quantitative evaluations of public policy programs. The programs usually have as their main targets firms or individuals who benefit from direct subsidies and/or training. It is hypothesised that because of the technical nature of quantitative evaluations, some of the public officials to whom these evaluations are intended, may find them too complex to comprehend fully. Hence, those officials might disregard them up front, or form a biased opinion (positive or negative) or even accept the results on their face value. However, because all evaluations are subjective by definition, the public officials should have some basic knowledge on the logic behind the design and context of evaluations. Only then, can they judge themselves on their worth, and consequently decide to what degree they will take into account their findings and recommendations. The paper initially discusses the issues of accountability and causality and then introduces policy evaluation as a two phase process: First, estimations are made on the potential impact of the policy in question and then a judgement is passed on the worth of the impacts estimated, through a cost benefit analysis. The estimations in turn, comprise of two related areas: the design of the evaluation and the model specification. In designs, one has to consider whether counterfactual populations are included or not and whether the impact variables are in cross-sectional or longitudinal format. In model specifications the evaluator must decide which independent control variables he will include in the regression model so as to account for selection bias. In cost benefit analysis decisions have to be made as to whether the analysis will be made at partial equilibrium or general equilibrium level and whether the judgements formulated will be based purely on efficiency grounds or using just distributional criteria as well. The paper recommends among others, that (a) public policy evaluations should establish clear rules of causation between the public intervention and the potential impact measured, (b) limitations in the estimation and cost benefit analysis phase must be explicitly stated and (c) retrospective evaluations should be conducted at closer intervals after the end of the intervention so as to reduce the external heterogeneity generated due to the time lag between the results produced and the on-going programs.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL:
Download Restriction: no

File URL:
Download Restriction: no

Bibliographic Info

Paper provided by Government Institute for Economic Research Finland (VATT) in its series Research Reports with number 90.

as in new window
Date of creation: 31 Jul 2002
Date of revision:
Handle: RePEc:fer:resrep:90

Contact details of provider:
Postal: Arkadiankatu 7, P.O. Box 1279, FI-00101 Helsinki
Phone: +358 295 519 400
Fax: +358 295 519 599
Web page:
More information through EDIRC

Order Information:

Related research

Keywords: Public policy evaluation; quantitative methods; cost benefit analysis;

Find related papers by JEL classification:


References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Jeffrey Smith & Petra Todd, 2003. "Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators?," University of Western Ontario, CIBC Centre for Human Capital and Productivity Working Papers, University of Western Ontario, CIBC Centre for Human Capital and Productivity 20035, University of Western Ontario, CIBC Centre for Human Capital and Productivity.
  2. James J. Heckman & Jeffrey A. Smith, 1998. "Evaluating the Welfare State," NBER Working Papers 6542, National Bureau of Economic Research, Inc.
  3. Richard Blundell & Monica Costa Dias, 2000. "Evaluation methods for non-experimental data," Fiscal Studies, Institute for Fiscal Studies, Institute for Fiscal Studies, vol. 21(4), pages 427-468, January.
  4. Schmidt, Christoph M, 2001. "Knowing What Works. The Case for Rigorous Programme Evaluation," CEPR Discussion Papers, C.E.P.R. Discussion Papers 2826, C.E.P.R. Discussion Papers.
  5. Heckman, James J, 1979. "Sample Selection Bias as a Specification Error," Econometrica, Econometric Society, Econometric Society, vol. 47(1), pages 153-61, January.
  6. Jochen Kluve & Hartmut Lehmann & Christoph M. Schmidt, 2000. "Disentangling Treatment Effects of Polish Active Labour Market Policies: Evidence from Matched Samples," CERT Discussion Papers, Centre for Economic Reform and Transformation, Heriot Watt University 0007, Centre for Economic Reform and Transformation, Heriot Watt University.
  7. Devarajan, Shantayanan & Squire, Lyn & Suthiwart-Narueput, Sethaput, 1997. "Beyond Rate of Return: Reorienting Project Appraisal," World Bank Research Observer, World Bank Group, World Bank Group, vol. 12(1), pages 35-46, February.
  8. Rajeev H. Dehejia & Sadek Wahba, 2002. "Propensity Score-Matching Methods For Nonexperimental Causal Studies," The Review of Economics and Statistics, MIT Press, MIT Press, vol. 84(1), pages 151-161, February.
  9. Jeffrey Smith, 2000. "A Critical Survey of Empirical Methods for Evaluating Active Labor Market Policies," UWO Department of Economics Working Papers, University of Western Ontario, Department of Economics 20006, University of Western Ontario, Department of Economics.
  10. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, American Economic Association, vol. 76(4), pages 604-20, September.
  11. Michael Lechner, 2002. "Program Heterogeneity And Propensity Score Matching: An Application To The Evaluation Of Active Labor Market Policies," The Review of Economics and Statistics, MIT Press, MIT Press, vol. 84(2), pages 205-220, May.
  12. Takis Venetoklis, 2001. "Business Subsidies and Bureaucratic Behaviour," Research Reports, Government Institute for Economic Research Finland (VATT) 79, Government Institute for Economic Research Finland (VATT).
  13. James J. Heckman, 2000. "Causal Parameters And Policy Analysis In Economics: A Twentieth Century Retrospective," The Quarterly Journal of Economics, MIT Press, MIT Press, vol. 115(1), pages 45-97, February.
  14. Heckman, James J & Ichimura, Hidehiko & Todd, Petra, 1998. "Matching as an Econometric Evaluation Estimator," Review of Economic Studies, Wiley Blackwell, Wiley Blackwell, vol. 65(2), pages 261-94, April.
  15. Lechner, Michael, 1999. "Identification and Estimation of Causal Effects of Multiple Treatments Under the Conditional Independence Assumption," IZA Discussion Papers 91, Institute for the Study of Labor (IZA).
  16. Takis Venetoklis & Aki Kangasharju, 2002. "Business Subsidies and Employment of Firms: Overall Evaluation and Regional Extension," Discussion Papers, Government Institute for Economic Research Finland (VATT) 268, Government Institute for Economic Research Finland (VATT).
  17. Mark Schreiner, 2001. "Evaluation and Microenterprise Programs," Development and Comp Systems, EconWPA 0108002, EconWPA, revised 27 Dec 2001.
  18. Heckman, James, 2001. "Accounting for Heterogeneity, Diversity and General Equilibrium in Evaluating Social Programmes," Economic Journal, Royal Economic Society, Royal Economic Society, vol. 111(475), pages F654-99, November.
  19. repec:fth:coluec:9899-01 is not listed on IDEAS
  20. McCloskey, Donald N, 1983. "The Rhetoric of Economics," Journal of Economic Literature, American Economic Association, American Economic Association, vol. 21(2), pages 481-517, June.
Full references (including those not matched with items on IDEAS)


Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. Anna Baranowska-Rataj & Anna Matysiak, 2012. "Czy znamy lekarstwo na nisk¹ dzietnoœæ? Wyniki miêdzynarodowych badañ ewaluacyjnych na temat polityki rodzinnej," Working Papers, Institute of Statistics and Demography, Warsaw School of Economics 47, Institute of Statistics and Demography, Warsaw School of Economics.


This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.


Access and download statistics


When requesting a correction, please mention this item's handle: RePEc:fer:resrep:90. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Anita Niskanen).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.