IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Log in (now much improved!) to save this article

Alternative Approaches to Evaluation in Empirical Microeconomics

  • Richard Blundell
  • Monica Costa Dias

This paper reviews some of the most popular policy evaluation methods in empirical microeconomics: social experiments, natural experiments, matching, instrumental variables, discontinuity design, and control functions. It discusses identification of traditionally used average parameters and more complex distributional parameters. The adequacy, assumptions, and data requirements of each approach are discussed, drawing on empirical evidence from the education and employment policy evaluation literature. A workhorse simulation model of education and earnings is used throughout the paper to discuss and illustrate each approach. The full set of STATA data sets and do-files are available free online and can be used to reproduce all estimation results.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://jhr.uwpress.org/cgi/reprint/44/3/565
Download Restriction: A subscription is required to access pdf files. Pay per article is available.

As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

Article provided by University of Wisconsin Press in its journal Journal of Human Resources.

Volume (Year): 44 (2009)
Issue (Month): 3 ()
Pages:

as
in new window

Handle: RePEc:uwp:jhriss:v:44:y:2009:i3:p565-640
Contact details of provider: Web page: http://jhr.uwpress.org/

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Austan Goolsbee, 1997. "What Happens When You Tax the Rich? Evidence from Executive Compensation," NBER Working Papers 6333, National Bureau of Economic Research, Inc.
  2. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
  3. Gary Burtless, 1985. "Are Targeted Wage Subsidies Harmful? Evidence from a Wage Voucher Experiment," ILR Review, Cornell University, ILR School, vol. 39(1), pages 105-114, October.
  4. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Publishing House "SINERGIA PRESS", vol. 31(3), pages 129-137.
  5. Blundell, Richard & Macurdy, Thomas, 1999. "Labor supply: A review of alternative approaches," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 27, pages 1559-1695 Elsevier.
  6. Ashenfelter, Orley & Card, David, 1985. "Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs," The Review of Economics and Statistics, MIT Press, vol. 67(4), pages 648-60, November.
  7. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097 Elsevier.
  8. Ahn, Hyungtaik & Powell, James L., 1993. "Semiparametric estimation of censored selection models with a nonparametric selection mechanism," Journal of Econometrics, Elsevier, vol. 58(1-2), pages 3-29, July.
  9. Frölich, Markus, 2006. "A Note on Parametric and Nonparametric Regression in the Presence of Endogenous Control Variables," IZA Discussion Papers 2126, Institute for the Study of Labor (IZA).
  10. Richard Blundell & Alan Duncan & Costas Meghir, 1995. "Estimating labour supply responses using tax reforms," IFS Working Papers W95/07, Institute for Fiscal Studies.
  11. Richard Blundell & Lorraine Dearden & Barbara Sianesi, 2004. "Evaluating the Impact of Education on Earnings in the UK: Models, Methods and Results from the NCDS," CEE Discussion Papers 0047, Centre for the Economics of Education, LSE.
  12. Bjorklund, Anders & Moffitt, Robert, 1987. "The Estimation of Wage Gains and Welfare Gains in Self-selection," The Review of Economics and Statistics, MIT Press, vol. 69(1), pages 42-49, February.
  13. Heckman, James & Navarro-Lozano, Salvador, 2003. "Using matching, instrumental variables and control functions to estimate economic choice models," Working Paper Series 2003:4, IFAU - Institute for Evaluation of Labour Market and Education Policy.
  14. Wilbert van der Klaauw, 2008. "Regression-Discontinuity Analysis: A Survey of Recent Developments in Economics," LABOUR, CEIS, vol. 22(2), pages 219-245, 06.
  15. James Heckman & Edward Vytlacil, 1998. "Instrumental Variables Methods for the Correlated Random Coefficient Model: Estimating the Average Rate of Return to Schooling When the Return is Correlated with Schooling," Journal of Human Resources, University of Wisconsin Press, vol. 33(4), pages 974-987.
  16. Dewatripont,Mathias & Hansen,Lars Peter & Turnovsky,Stephen J. (ed.), 2003. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9780521818735, November.
  17. Carneiro, Pedro & Hansen, Karsten T & Heckman, James J, 2002. "Removing the veil of ignorance in assessing the distributional impacts of social policies," Working Paper Series 2002:2, IFAU - Institute for Evaluation of Labour Market and Education Policy.
  18. Whitney Newey & James Powell & Francis Vella, 1998. "Nonparametric Estimation of Triangular Simultaneous Equations Models," Working papers 98-16, Massachusetts Institute of Technology (MIT), Department of Economics.
  19. Nada Eissa & Jeffrey B. Liebman, 1996. "Labor Supply Response to the Earned Income Tax Credit," The Quarterly Journal of Economics, Oxford University Press, vol. 111(2), pages 605-637.
  20. Susan Athey & Guido W. Imbens, 2006. "Identification and Inference in Nonlinear Difference-in-Differences Models," Econometrica, Econometric Society, vol. 74(2), pages 431-497, 03.
  21. David Card & Lara D. Shore-Sheppard, 2003. "Using Discontinuous Eligibility Rules to Identify the Effects of the Federal Medicaid Expansions on Low Income Children," Department of Economics Working Papers 2003-09, Department of Economics, Williams College.
  22. Lindsey, Lawrence B., 1987. "Individual taxpayer response to tax cuts: 1982-1984 : With implications for the revenue maximizing tax rate," Journal of Public Economics, Elsevier, vol. 33(2), pages 173-206, July.
  23. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February.
  24. Alberto Abadie & Joshua Angrist & Guido Imbens, 2002. "Instrumental Variables Estimates of the Effect of Subsidized Training on the Quantiles of Trainee Earnings," Econometrica, Econometric Society, vol. 70(1), pages 91-117, January.
  25. Arulampalam, W. & Robin A. Naylor & Jeremy P. Smith, 2002. "University of Warwick," Royal Economic Society Annual Conference 2002 9, Royal Economic Society.
  26. Richard Blundell & James Powell, 2001. "Endogeneity in semiparametric binary response models," CeMMAP working papers CWP05/01, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
  27. Joshua D. Angrist & Guido W. Imbens, 1995. "Identification and Estimation of Local Average Treatment Effects," NBER Technical Working Papers 0118, National Bureau of Economic Research, Inc.
  28. Horowitz, Joel L., 2001. "The Bootstrap," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 5, chapter 52, pages 3159-3228 Elsevier.
  29. Feldstein, Martin, 1995. "Effect of Marginal Tax Rates on Taxable Income: A Panel Study of the 1986 Tax Reform Act," Scholarly Articles 2766676, Harvard University Department of Economics.
  30. Feldstein, Martin, 1995. "The Effect of Marginal Tax Rates on Taxable Income: A Panel Study of the 1986 Tax Reform Act," Journal of Political Economy, University of Chicago Press, vol. 103(3), pages 551-72, June.
  31. Sascha O. Becker & Andrea Ichino, 2002. "Estimation of average treatment effects based on propensity scores," Stata Journal, StataCorp LP, vol. 2(4), pages 358-377, November.
  32. Bassi, Laurie J, 1984. "Estimating the Effect of Training Programs with Non-Random Selection," The Review of Economics and Statistics, MIT Press, vol. 66(1), pages 36-43, February.
  33. Donald W. K. Andrews & Marcia M. A. Schafgans, 1998. "Semiparametric Estimation of the Intercept of a Sample Selection Model," Review of Economic Studies, Oxford University Press, vol. 65(3), pages 497-517.
  34. Dewatripont,Mathias & Hansen,Lars Peter & Turnovsky,Stephen J. (ed.), 2003. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9780521818742, November.
  35. Hahn, Jinyong & Todd, Petra & Van der Klaauw, Wilbert, 2001. "Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design," Econometrica, Econometric Society, vol. 69(1), pages 201-09, January.
  36. Barbara Sianesi, 2001. "Propensity score matching," United Kingdom Stata Users' Group Meetings 2001 12, Stata Users Group, revised 23 Aug 2001.
  37. James J. Heckman & Hidehiko Ichimura & Petra Todd, 1998. "Matching As An Econometric Evaluation Estimator," Review of Economic Studies, Oxford University Press, vol. 65(2), pages 261-294.
  38. James J. Heckman & Edward J. Vytlacil, 2000. "Local Instrumental Variables," NBER Technical Working Papers 0252, National Bureau of Economic Research, Inc.
  39. Jerry A. Hausman & David A. Wise, 1985. "Social Experimentation," NBER Books, National Bureau of Economic Research, Inc, number haus85-1, May.
  40. Nickell, Stephen J, 1981. "Biases in Dynamic Models with Fixed Effects," Econometrica, Econometric Society, vol. 49(6), pages 1417-26, November.
  41. Jeffrey Smith & Petra Todd, 2003. "Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators?," University of Western Ontario, Centre for Human Capital and Productivity (CHCP) Working Papers 20035, University of Western Ontario, Centre for Human Capital and Productivity (CHCP).
  42. Blundell, Richard William & Costa Dias, Monica & Meghir, Costas & Van Reenen, John, 2003. "Evaluating the Employment Impact of a Mandatory Job Search Programme," CEPR Discussion Papers 3786, C.E.P.R. Discussion Papers.
  43. Mitali Das & Whitney K. Newey & Francis Vella, 2003. "Nonparametric Estimation of Sample Selection Models," Review of Economic Studies, Oxford University Press, vol. 70(1), pages 33-58.
  44. Feenberg, D.R. & Poterba, J.M., 1992. "Income Inequality and the Incomes of Very High Income Taxpayers: Evidence from Tax Returns," Working papers 92-16, Massachusetts Institute of Technology (MIT), Department of Economics.
  45. Jinyong Hahn, 1998. "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica, Econometric Society, vol. 66(2), pages 315-332, March.
  46. Dewatripont,Mathias & Hansen,Lars Peter & Turnovsky,Stephen J. (ed.), 2003. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9780521524124, November.
  47. Dewatripont,Mathias & Hansen,Lars Peter & Turnovsky,Stephen J. (ed.), 2003. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9780521524117, November.
  48. James J. Heckman & Jeffrey Smith & Nancy Clements, 1997. "Making The Most Out Of Programme Evaluations and Social Experiments: Accounting For Heterogeneity in Programme Impacts," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 487-535.
  49. Alberto Abadie & Joshua D. Angrist & Guido W. Imbens, 1998. "Instrumental Variables Estimation of Quantile Treatment Effects," NBER Technical Working Papers 0229, National Bureau of Economic Research, Inc.
  50. Robert Moffit, 2007. "Estimating Marginal Treatment Effects in Heterogeneous Populations," Economics Working Paper Archive 539, The Johns Hopkins University,Department of Economics.
  51. Richard Blundell & Monica Costa Dias, 2000. "Evaluation methods for non-experimental data," Fiscal Studies, Institute for Fiscal Studies, vol. 21(4), pages 427-468, January.
  52. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," NBER Working Papers 6699, National Bureau of Economic Research, Inc.
  53. Edwin Leuven & Barbara Sianesi, 2003. "PSMATCH2: Stata module to perform full Mahalanobis and propensity score matching, common support graphing, and covariate imbalance testing," Statistical Software Components S432001, Boston College Department of Economics, revised 19 Jan 2015.
  54. Richard Blundell & Monica Costa Dias & Costas Meghir & John Van Reenen, 2001. "Evaluating the employment impact of a mandatory job search assistance program," IFS Working Papers W01/20, Institute for Fiscal Studies.
  55. Jaap H. Abbring & Gerard J. van den Berg, 2003. "The Nonparametric Identification of Treatment Effects in Duration Models," Econometrica, Econometric Society, vol. 71(5), pages 1491-1517, 09.
  56. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-20, September.
  57. Dewatripont,Mathias & Hansen,Lars Peter & Turnovsky,Stephen J. (ed.), 2003. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9780521818728, November.
  58. Richard Blundell & James Powell, 2001. "Endogeneity in nonparametric and semiparametric regression models," CeMMAP working papers CWP09/01, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
  59. Lee, David S., 2008. "Randomized experiments from non-random selection in U.S. House elections," Journal of Econometrics, Elsevier, vol. 142(2), pages 675-697, February.
  60. James J. Heckman & Jeffrey A. Smith, 1999. "The Pre-Program Earnings Dip and the Determinants of Participation in a Social Program: Implications for Simple Program Evaluation Strategies," NBER Working Papers 6983, National Bureau of Economic Research, Inc.
  61. Battistin, Erich & Rettore, Enrico, 2008. "Ineligibles and eligible non-participants as a double comparison group in regression-discontinuity designs," Journal of Econometrics, Elsevier, vol. 142(2), pages 715-730, February.
  62. Hausman, Jerry A. & Wise, David A. (ed.), 1985. "Social Experimentation," National Bureau of Economic Research Books, University of Chicago Press, edition 0, number 9780226319407.
  63. Jerry A. Hausman & David A. Wise, 1985. "Introduction to "Social Experimentation"," NBER Chapters, in: Social Experimentation, pages 1-10 National Bureau of Economic Research, Inc.
  64. Heckman, James J, 1990. "Varieties of Selection Bias," American Economic Review, American Economic Association, vol. 80(2), pages 313-18, May.
  65. Ten Have, Thomas R. & Elliott, Michael R. & Joffe, Marshall & Zanutto, Elaine & Datto, Catherine, 2004. "Causal Models for Randomized Physician Encouragement Trials in Treating Primary Care Depression," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 16-25, January.
  66. Dewatripont,Mathias & Hansen,Lars Peter & Turnovsky,Stephen J. (ed.), 2003. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9780521524131, November.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:uwp:jhriss:v:44:y:2009:i3:p565-640. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ()

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.