Matching vs Differencing when Estimating Treatment Effects with Panel Data: the Example of the Effect of Job Training Programs on Earnings
This paper compares matching and Difference-In-Difference matching (DID) when estimating the effect of a program on a dynamic outcome. I detail the sources of bias of each estimator in a model of entry into a Job Training Program (JTP) and earnings dynamics that I use as a working example. I show that there are plausible settings in which DID is consistent while matching on past outcomes is not. Unfortunately, the consistency of both estimators relies on conditions that are at odds with properties of earnings dynamics. Using calibration and Monte-Carlo simulations, I show that deviations from the most favorable conditions severely bias both estimators. The behavior of matching is nevertheless less erratic: its bias generally decreases when controlling for more past outcomes and it generally provides a lower bound on the true treatment effect. I finally point to previously unnoticed empirical results that confirm that DID does well, and generally better than matching on past outcomes, at replicating the results of an experimental benchmark.
|Date of creation:||Oct 2012|
|Date of revision:|
|Publication status:||Published in Journal of Econometrics, vol.�185, n°1, mars 2015, p.�110-123.|
|Contact details of provider:|| Phone: (+33) 5 61 12 86 23|
Web page: http://www.tse-fr.eu/
More information through EDIRC
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Wooldridge, Jeffrey M., 2005. "Violating Ignorability Of Treatment By Controlling For Too Many Factors," Econometric Theory, Cambridge University Press, vol. 21(05), pages 1026-1028, October.
- Nicolas R. Ziebarth & Martin Karlsson, 2009.
"A Natural Experiment on Sick Pay Cuts, Sickness Absence, and Labor Costs,"
SOEPpapers on Multidisciplinary Panel Data Research
244, DIW Berlin, The German Socio-Economic Panel (SOEP).
- Ziebarth, Nicolas R. & Karlsson, Martin, 2010. "A natural experiment on sick pay cuts, sickness absence, and labor costs," Journal of Public Economics, Elsevier, vol. 94(11-12), pages 1108-1122, December.
- Ziebarth N & Karlsson M, 2009. "A Natural Experiment on Sick Pay Cuts, Sickness Absence, and Labor Costs," Health, Econometrics and Data Group (HEDG) Working Papers 09/34, HEDG, c/o Department of Economics, University of York.
- Jeffrey Smith & Petra Todd, 2003.
"Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators?,"
University of Western Ontario, CIBC Centre for Human Capital and Productivity Working Papers
20035, University of Western Ontario, CIBC Centre for Human Capital and Productivity.
- A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
- Puhani, Patrick A. & Sonderhof, Katja, 2010.
"The effects of a sick pay reform on absence and on health-related outcomes,"
Journal of Health Economics,
Elsevier, vol. 29(2), pages 285-302, March.
- Puhani, Patrick A. & Sonderhof, Katja, 2009. "The Effects of a Sick Pay Reform on Absence and on Health-Related Outcomes," IZA Discussion Papers 4607, Institute for the Study of Labor (IZA).
- Patrick A. Puhani & Katja Sonderhof, 2009. "The Effects of a Sick Pay Reform on Absence and on Health-Related Outcomes," SOEPpapers on Multidisciplinary Panel Data Research 248, DIW Berlin, The German Socio-Economic Panel (SOEP).
- Patrick Puhani & Katja Sonderhof, 2009. "The Effects of a Sick Pay Reform on Absence and on Health-Related Outcomes," University of St. Gallen Department of Economics working paper series 2009 2009-34, Department of Economics, University of St. Gallen.
- Javier Alvarez & Martin Browning & Mette Ejrnæs, 2001.
"Modelling Income Processes with lots of heterogeneity,"
CAM Working Papers
2002-01, University of Copenhagen. Department of Economics. Centre for Applied Microeconometrics.
- Martin Browning & Mette Ejrn�s & Javier Alvarez, 2010. "Modelling Income Processes with Lots of Heterogeneity," Review of Economic Studies, Oxford University Press, vol. 77(4), pages 1353-1381.
- Javier Alvarez & Martin Browning & Mette Ejrnæs, 2002. "Modelling income processes with lots of heterogeneity," 10th International Conference on Panel Data, Berlin, July 5-6, 2002 D2-3, International Conferences on Panel Data.
- Martin Browning & Mette Ejrnaes, 2006. "Modelling income processes with lots of heterogeneity," Economics Series Working Papers 285, University of Oxford, Department of Economics.
- James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998.
"Characterizing Selection Bias Using Experimental Data,"
Econometric Society, vol. 66(5), pages 1017-1098, September.
- James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," NBER Working Papers 6699, National Bureau of Economic Research, Inc.
- Fatih Guvenen, 2006.
"Learning your earning: are labor income shocks really very persistent?,"
Discussion Paper / Institute for Empirical Macroeconomics
145, Federal Reserve Bank of Minneapolis.
- Fatih Guvenen, 2007. "Learning Your Earning: Are Labor Income Shocks Really Very Persistent?," American Economic Review, American Economic Association, vol. 97(3), pages 687-712, June.
- Fatih Guvenen, 2005. "Learning Your Earning: Are Labor Income Shocks Really Very Persistent?," Macroeconomics 0507004, EconWPA.
- Jay Bhattacharya & William B. Vogt, 2007. "Do Instrumental Variables Belong in Propensity Scores?," NBER Technical Working Papers 0343, National Bureau of Economic Research, Inc.
- MaCurdy, Thomas E., 1982. "The use of time series processes to model the error structure of earnings in a longitudinal data analysis," Journal of Econometrics, Elsevier, vol. 18(1), pages 83-114, January.
- Markus Frölich, 2004. "Finite-Sample Properties of Propensity-Score Matching and Weighting Estimators," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 77-90, February.
- Barry Arnold & Robert Beaver & Richard Groeneveld & William Meeker, 1993. "The nontruncated marginal of a truncated bivariate normal distribution," Psychometrika, Springer, vol. 58(3), pages 471-488, September.
When requesting a correction, please mention this item's handle: RePEc:tse:wpaper:26567. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ()
If references are entirely missing, you can add them using this form.