IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Log in (now much improved!) to save this paper

New Evidence on Linear Regression and Treatment Effect Heterogeneity

Listed author(s):
  • Sloczynski, Tymon

    ()

    (Brandeis University)

It is standard practice in applied work to rely on linear least squares regression to estimate the effect of a binary variable ("treatment") on some outcome of interest. In this paper I study the interpretation of the regression estimand when treatment effects are in fact heterogeneous. I show that the coefficient on treatment is identical to the outcome of the following three-step procedure: first, calculate the linear projection of treatment on the vector of other covariates ("propensity score"); second, calculate average partial effects for both groups of interest ("treated" and "controls") from a regression of outcome on treatment, the propensity score, and their interaction; third, calculate a weighted average of these two effects, with weights being inversely related to the unconditional probability that a unit belongs to a given group. Each of these steps is potentially problematic, but this last property – the reliance on implicit weights which are inversely related to the proportion of each group – can have particularly severe consequences for applied work. To illustrate the importance of this result, I perform Monte Carlo simulations as well as replicate two applied papers: Berger, Easterly, Nunn and Satyanath (2013) on the effects of successful CIA interventions during the Cold War on imports from the US; and Martinez-Bravo (2014) on the effects of appointed officials on village-level electoral results in Indonesia. In both cases some of the conclusions change dramatically after allowing for heterogeneity in effects.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://ftp.iza.org/dp9491.pdf
Download Restriction: no

Paper provided by Institute for the Study of Labor (IZA) in its series IZA Discussion Papers with number 9491.

as
in new window

Length: 44 pages
Date of creation: Nov 2015
Handle: RePEc:iza:izadps:dp9491
Contact details of provider: Postal:
IZA, P.O. Box 7240, D-53072 Bonn, Germany

Phone: +49 228 3894 223
Fax: +49 228 3894 180
Web page: http://www.iza.org

Order Information: Postal: IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany
Email:


References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as
in new window


  1. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
  2. Marianne P. Bitler & Jonah B. Gelbach & Hilary W. Hoynes, 2006. "What Mean Impacts Miss: Distributional Effects of Welfare Reform Experiments," American Economic Review, American Economic Association, vol. 96(4), pages 988-1012, September.
  3. James J. Heckman & Vytlacil, Edward J., 2007. "Econometric Evaluation of Social Programs, Part II: Using the Marginal Treatment Effect to Organize Alternative Econometric Estimators to Evaluate Social Programs, and to Forecast their Effects in New," Handbook of Econometrics,in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 71 Elsevier.
  4. Monica Martinez-Bravo, 2014. "The Role of Local Officials in New Democracies: Evidence from Indonesia," American Economic Review, American Economic Association, vol. 104(4), pages 1244-1287, April.
  5. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2008. "Nonparametric Tests for Treatment Effect Heterogeneity," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 389-405, August.
  6. Sergio Firpo, 2007. "Efficient Semiparametric Estimation of Quantile Treatment Effects," Econometrica, Econometric Society, vol. 75(1), pages 259-276, January.
  7. James J. Heckman & Edward Vytlacil, 2005. "Structural Equations, Treatment Effects, and Econometric Policy Evaluation," Econometrica, Econometric Society, vol. 73(3), pages 669-738, May.
  8. Maury Gittleman & Edward N. Wolff, 2004. "Racial Differences in Patterns of Wealth Accumulation," Journal of Human Resources, University of Wisconsin Press, vol. 39(1).
  9. Richard Blundell & Monica Costa Dias, 2009. "Alternative Approaches to Evaluation in Empirical Microeconomics," Journal of Human Resources, University of Wisconsin Press, vol. 44(3).
  10. Markus Frölich & Blaise Melly, 2013. "Unconditional Quantile Treatment Effects Under Endogeneity," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 31(3), pages 346-357, July.
  11. Elder, Todd E. & Goddeeris, John H. & Haider, Steven J., 2010. "Unexplained gaps and Oaxaca-Blinder decompositions," Labour Economics, Elsevier, vol. 17(1), pages 284-290, January.
  12. Alberto Alesina & Paola Giuliano & Nathan Nunn, 2013. "On the Origins of Gender Roles: Women and the Plough," The Quarterly Journal of Economics, Oxford University Press, vol. 128(2), pages 469-530.
  13. Timothy N. Bond & Kevin Lang, 2013. "The Evolution of the Black-White Test Score Gap in Grades K–3: The Fragility of Results," The Review of Economics and Statistics, MIT Press, vol. 95(5), pages 1468-1479, December.
  14. Katrine V. Løken & Magne Mogstad & Matthew Wiswall, 2012. "What Linear Estimators Miss: The Effects of Family Income on Child Outcomes," American Economic Journal: Applied Economics, American Economic Association, vol. 4(2), pages 1-35, April.
  15. Matias Busso & John DiNardo & Justin McCrary, 2014. "New Evidence on the Finite Sample Properties of Propensity Score Reweighting and Matching Estimators," The Review of Economics and Statistics, MIT Press, vol. 96(5), pages 885-897, December.
  16. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, 07.
  17. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, January.
  18. Fortin, Nicole & Lemieux, Thomas & Firpo, Sergio, 2011. "Decomposition Methods in Economics," Handbook of Labor Economics, Elsevier.
  19. Roland G. Fryer & Michael Greenstone, 2010. "The Changing Consequences of Attending Historically Black Colleges and Universities," American Economic Journal: Applied Economics, American Economic Association, vol. 2(1), pages 116-148, January.
  20. Ben Jann, 2008. "The Blinder–Oaxaca decomposition for linear regression models," Stata Journal, StataCorp LP, vol. 8(4), pages 453-479, December.
  21. Oaxaca, Ronald, 1973. "Male-Female Wage Differentials in Urban Labor Markets," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 14(3), pages 693-709, October.
  22. Kevin Lang & Michael Manove, 2011. "Education and Labor Market Discrimination," American Economic Review, American Economic Association, vol. 101(4), pages 1467-1496, June.
  23. repec:pri:rpdevs:deaton_instruments_randomization_learning_all_04april_2010 is not listed on IDEAS
  24. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
  25. Bitler, Marianne P. & Gelbach, Jonah B. & Hoynes, Hilary W., 2008. "Distributional impacts of the Self-Sufficiency Project," Journal of Public Economics, Elsevier, vol. 92(3-4), pages 748-765, April.
  26. Barsky R. & Bound J. & Charles K.K. & Lupton J.P., 2002. "Accounting for the Black-White Wealth Gap: A Nonparametric Approach," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 663-673, September.
  27. Djebbari, Habiba & Smith, Jeffrey, 2008. "Heterogeneous impacts in PROGRESA," Journal of Econometrics, Elsevier, vol. 145(1-2), pages 64-80, July.
  28. repec:mpr:mprres:6573 is not listed on IDEAS
  29. Jesse Rothstein & Nathan Wozny, 2013. "Permanent Income and the Black-White Test Score Gap," Journal of Human Resources, University of Wisconsin Press, vol. 48(3), pages 510-544.
  30. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
  31. James J. Heckman & Sergio Urzua & Edward Vytlacil, 2006. "Understanding Instrumental Variables in Models with Essential Heterogeneity," The Review of Economics and Statistics, MIT Press, vol. 88(3), pages 389-432, August.
  32. Guido W. Imbens, 2015. "Matching Methods in Practice: Three Examples," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 373-419.
  33. Jeffrey M. Wooldridge, 2005. "Fixed-Effects and Related Estimators for Correlated Random-Coefficient and Treatment-Effect Panel Data Models," The Review of Economics and Statistics, MIT Press, vol. 87(2), pages 385-390, May.
  34. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
  35. Kline, Patrick, 2014. "A note on variance estimation for the Oaxaca estimator of average treatment effects," Economics Letters, Elsevier, vol. 122(3), pages 428-431.
  36. Alan S. Blinder, 1973. "Wage Discrimination: Reduced Form and Structural Estimates," Journal of Human Resources, University of Wisconsin Press, vol. 8(4), pages 436-455.
  37. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
  38. Patrick Kline, 2011. "Oaxaca-Blinder as a Reweighting Estimator," American Economic Review, American Economic Association, vol. 101(3), pages 532-537, May.
  39. Douglas Almond & Kenneth Y. Chay & David S. Lee, 2005. "The Costs of Low Birth Weight," The Quarterly Journal of Economics, Oxford University Press, vol. 120(3), pages 1031-1083.
  40. James J. Heckman, 2001. "Micro Data, Heterogeneity, and the Evaluation of Public Policy: Nobel Lecture," Journal of Political Economy, University of Chicago Press, vol. 109(4), pages 673-748, August.
  41. Gary Solon & Steven J. Haider & Jeffrey M. Wooldridge, 2015. "What Are We Weighting For?," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 301-316.
  42. Peter Z. Schochet, 2010. "Is Regression Adjustment Supported By the Neyman Model for Causal Inference?," Mathematica Policy Research Reports 782da2242fba458eb61752f96, Mathematica Policy Research.
  43. Roland G. Fryer & Steven D. Levitt, 2010. "An Empirical Analysis of the Gender Gap in Mathematics," American Economic Journal: Applied Economics, American Economic Association, vol. 2(2), pages 210-240, April.
  44. Ahmed Khwaja & Gabriel Picone & Martin Salm & Justin G. Trogdon, 2011. "A comparison of treatment effects estimators using a structural model of AMI treatment choices and severity of illness information from hospital charts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(5), pages 825-853, August.
  45. Joshua D. Angrist, 1998. "Estimating the Labor Market Impact of Voluntary Military Service Using Social Security Data on Military Applicants," Econometrica, Econometric Society, vol. 66(2), pages 249-288, March.
  46. Victor Chernozhukov & Iván Fernández‐Val & Jinyong Hahn & Whitney Newey, 2013. "Average and Quantile Effects in Nonseparable Panel Models," Econometrica, Econometric Society, vol. 81(2), pages 535-580, March.
  47. Daniel Berger & William Easterly & Nathan Nunn & Shanker Satyanath, 2013. "Commercial Imperialism? Political Influence and Trade during the Cold War," American Economic Review, American Economic Association, vol. 103(2), pages 863-896, April.
  48. Joshua D. Angrist & Kathryn Graddy & Guido W. Imbens, 2000. "The Interpretation of Instrumental Variables Estimators in Simultaneous Equations Models with an Application to the Demand for Fish," Review of Economic Studies, Oxford University Press, vol. 67(3), pages 499-527.
  49. Roland G. Fryer & Steven D. Levitt, 2004. "Understanding the Black-White Test Score Gap in the First Two Years of School," The Review of Economics and Statistics, MIT Press, vol. 86(2), pages 447-464, May.
  50. repec:hrv:faseco:33077826 is not listed on IDEAS
  51. William Rhodes, 2010. "Heterogeneous Treatment Effects: What Does a Regression Estimate?," Evaluation Review, , vol. 34(4), pages 334-361, August.
  52. Dan A. Black & Jeffrey A. Smith & Mark C. Berger & Brett J. Noel, 2003. "Is the Threat of Reemployment Services More Effective Than the Services Themselves? Evidence from Random Assignment in the UI System," American Economic Review, American Economic Association, vol. 93(4), pages 1313-1327, September.
  53. Yitzhaki, Shlomo, 1996. "On Using Linear Regressions in Welfare Economics," Journal of Business & Economic Statistics, American Statistical Association, vol. 14(4), pages 478-486, October.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp9491. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Mark Fallak)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.