IDEAS home Printed from
   My bibliography  Save this paper

New Evidence on Linear Regression and Treatment Effect Heterogeneity


  • Sloczynski, Tymon

    () (Brandeis University)


It is standard practice in applied work to rely on linear least squares regression to estimate the effect of a binary variable ("treatment") on some outcome of interest. In this paper I study the interpretation of the regression estimand when treatment effects are in fact heterogeneous. I show that the coefficient on treatment is identical to the outcome of the following three-step procedure: first, calculate the linear projection of treatment on the vector of other covariates ("propensity score"); second, calculate average partial effects for both groups of interest ("treated" and "controls") from a regression of outcome on treatment, the propensity score, and their interaction; third, calculate a weighted average of these two effects, with weights being inversely related to the unconditional probability that a unit belongs to a given group. Each of these steps is potentially problematic, but this last property – the reliance on implicit weights which are inversely related to the proportion of each group – can have particularly severe consequences for applied work. To illustrate the importance of this result, I perform Monte Carlo simulations as well as replicate two applied papers: Berger, Easterly, Nunn and Satyanath (2013) on the effects of successful CIA interventions during the Cold War on imports from the US; and Martinez-Bravo (2014) on the effects of appointed officials on village-level electoral results in Indonesia. In both cases some of the conclusions change dramatically after allowing for heterogeneity in effects.

Suggested Citation

  • Sloczynski, Tymon, 2015. "New Evidence on Linear Regression and Treatment Effect Heterogeneity," IZA Discussion Papers 9491, Institute of Labor Economics (IZA).
  • Handle: RePEc:iza:izadps:dp9491

    Download full text from publisher

    File URL:
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    1. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    2. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    3. Marianne P. Bitler & Jonah B. Gelbach & Hilary W. Hoynes, 2006. "What Mean Impacts Miss: Distributional Effects of Welfare Reform Experiments," American Economic Review, American Economic Association, vol. 96(4), pages 988-1012, September.
    4. James J. Heckman & Vytlacil, Edward J., 2007. "Econometric Evaluation of Social Programs, Part II: Using the Marginal Treatment Effect to Organize Alternative Econometric Estimators to Evaluate Social Programs, and to Forecast their Effects in New," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.),Handbook of Econometrics, edition 1, volume 6, chapter 71, Elsevier.
    5. Monica Martinez-Bravo, 2014. "The Role of Local Officials in New Democracies: Evidence from Indonesia," American Economic Review, American Economic Association, vol. 104(4), pages 1244-1287, April.
    6. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2008. "Nonparametric Tests for Treatment Effect Heterogeneity," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 389-405, August.
    7. Sergio Firpo, 2007. "Efficient Semiparametric Estimation of Quantile Treatment Effects," Econometrica, Econometric Society, vol. 75(1), pages 259-276, January.
    8. Richard Blundell & Monica Costa Dias, 2009. "Alternative Approaches to Evaluation in Empirical Microeconomics," Journal of Human Resources, University of Wisconsin Press, vol. 44(3).
    9. James J. Heckman & Edward Vytlacil, 2005. "Structural Equations, Treatment Effects, and Econometric Policy Evaluation," Econometrica, Econometric Society, vol. 73(3), pages 669-738, May.
    10. Maury Gittleman & Edward N. Wolff, 2004. "Racial Differences in Patterns of Wealth Accumulation," Journal of Human Resources, University of Wisconsin Press, vol. 39(1).
    11. Elder, Todd E. & Goddeeris, John H. & Haider, Steven J., 2010. "Unexplained gaps and Oaxaca-Blinder decompositions," Labour Economics, Elsevier, vol. 17(1), pages 284-290, January.
    12. Markus Frölich & Blaise Melly, 2013. "Unconditional Quantile Treatment Effects Under Endogeneity," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 31(3), pages 346-357, July.
    13. Alberto Alesina & Paola Giuliano & Nathan Nunn, 2013. "On the Origins of Gender Roles: Women and the Plough," The Quarterly Journal of Economics, Oxford University Press, vol. 128(2), pages 469-530.
    14. Timothy N. Bond & Kevin Lang, 2013. "The Evolution of the Black-White Test Score Gap in Grades K–3: The Fragility of Results," The Review of Economics and Statistics, MIT Press, vol. 95(5), pages 1468-1479, December.
    15. Katrine V. Løken & Magne Mogstad & Matthew Wiswall, 2012. "What Linear Estimators Miss: The Effects of Family Income on Child Outcomes," American Economic Journal: Applied Economics, American Economic Association, vol. 4(2), pages 1-35, April.
    16. Matias Busso & John DiNardo & Justin McCrary, 2014. "New Evidence on the Finite Sample Properties of Propensity Score Reweighting and Matching Estimators," The Review of Economics and Statistics, MIT Press, vol. 96(5), pages 885-897, December.
    17. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, July.
    18. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, December.
    19. Fortin, Nicole & Lemieux, Thomas & Firpo, Sergio, 2011. "Decomposition Methods in Economics," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.),Handbook of Labor Economics, edition 1, volume 4, chapter 1, pages 1-102, Elsevier.
    20. Roland G. Fryer & Michael Greenstone, 2010. "The Changing Consequences of Attending Historically Black Colleges and Universities," American Economic Journal: Applied Economics, American Economic Association, vol. 2(1), pages 116-148, January.
    21. Ben Jann, 2008. "The Blinder–Oaxaca decomposition for linear regression models," Stata Journal, StataCorp LP, vol. 8(4), pages 453-479, December.
    22. Oaxaca, Ronald, 1973. "Male-Female Wage Differentials in Urban Labor Markets," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 14(3), pages 693-709, October.
    23. Kevin Lang & Michael Manove, 2011. "Education and Labor Market Discrimination," American Economic Review, American Economic Association, vol. 101(4), pages 1467-1496, June.
    24. repec:pri:rpdevs:deaton_instruments_randomization_learning_all_04april_2010 is not listed on IDEAS
    25. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    26. Barsky R. & Bound J. & Charles K.K. & Lupton J.P., 2002. "Accounting for the Black-White Wealth Gap: A Nonparametric Approach," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 663-673, September.
    27. Djebbari, Habiba & Smith, Jeffrey, 2008. "Heterogeneous impacts in PROGRESA," Journal of Econometrics, Elsevier, vol. 145(1-2), pages 64-80, July.
    28. repec:mpr:mprres:6573 is not listed on IDEAS
    29. Jesse Rothstein & Nathan Wozny, 2013. "Permanent Income and the Black-White Test Score Gap," Journal of Human Resources, University of Wisconsin Press, vol. 48(3), pages 510-544.
    30. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    31. James J. Heckman & Sergio Urzua & Edward Vytlacil, 2006. "Understanding Instrumental Variables in Models with Essential Heterogeneity," The Review of Economics and Statistics, MIT Press, vol. 88(3), pages 389-432, August.
    32. Jeffrey M. Wooldridge, 2005. "Fixed-Effects and Related Estimators for Correlated Random-Coefficient and Treatment-Effect Panel Data Models," The Review of Economics and Statistics, MIT Press, vol. 87(2), pages 385-390, May.
    33. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    34. Kline, Patrick, 2014. "A note on variance estimation for the Oaxaca estimator of average treatment effects," Economics Letters, Elsevier, vol. 122(3), pages 428-431.
    35. Alan S. Blinder, 1973. "Wage Discrimination: Reduced Form and Structural Estimates," Journal of Human Resources, University of Wisconsin Press, vol. 8(4), pages 436-455.
    36. Patrick Kline, 2011. "Oaxaca-Blinder as a Reweighting Estimator," American Economic Review, American Economic Association, vol. 101(3), pages 532-537, May.
    37. Bitler, Marianne P. & Gelbach, Jonah B. & Hoynes, Hilary W., 2008. "Distributional impacts of the Self-Sufficiency Project," Journal of Public Economics, Elsevier, vol. 92(3-4), pages 748-765, April.
    38. Douglas Almond & Kenneth Y. Chay & David S. Lee, 2005. "The Costs of Low Birth Weight," The Quarterly Journal of Economics, Oxford University Press, vol. 120(3), pages 1031-1083.
    39. James J. Heckman, 2001. "Micro Data, Heterogeneity, and the Evaluation of Public Policy: Nobel Lecture," Journal of Political Economy, University of Chicago Press, vol. 109(4), pages 673-748, August.
    40. Gary Solon & Steven J. Haider & Jeffrey M. Wooldridge, 2015. "What Are We Weighting For?," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 301-316.
    41. Peter Z. Schochet, "undated". "Is Regression Adjustment Supported By the Neyman Model for Causal Inference?," Mathematica Policy Research Reports 782da2242fba458eb61752f96, Mathematica Policy Research.
    42. Roland G. Fryer & Steven D. Levitt, 2010. "An Empirical Analysis of the Gender Gap in Mathematics," American Economic Journal: Applied Economics, American Economic Association, vol. 2(2), pages 210-240, April.
    43. Guido W. Imbens, 2015. "Matching Methods in Practice: Three Examples," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 373-419.
    44. Ahmed Khwaja & Gabriel Picone & Martin Salm & Justin G. Trogdon, 2011. "A comparison of treatment effects estimators using a structural model of AMI treatment choices and severity of illness information from hospital charts," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(5), pages 825-853, August.
    45. Joshua D. Angrist, 1998. "Estimating the Labor Market Impact of Voluntary Military Service Using Social Security Data on Military Applicants," Econometrica, Econometric Society, vol. 66(2), pages 249-288, March.
    46. Victor Chernozhukov & Iván Fernández‐Val & Jinyong Hahn & Whitney Newey, 2013. "Average and Quantile Effects in Nonseparable Panel Models," Econometrica, Econometric Society, vol. 81(2), pages 535-580, March.
    47. Daniel Berger & William Easterly & Nathan Nunn & Shanker Satyanath, 2013. "Commercial Imperialism? Political Influence and Trade during the Cold War," American Economic Review, American Economic Association, vol. 103(2), pages 863-896, April.
    48. Joshua D. Angrist & Kathryn Graddy & Guido W. Imbens, 2000. "The Interpretation of Instrumental Variables Estimators in Simultaneous Equations Models with an Application to the Demand for Fish," Review of Economic Studies, Oxford University Press, vol. 67(3), pages 499-527.
    49. Roland G. Fryer & Steven D. Levitt, 2004. "Understanding the Black-White Test Score Gap in the First Two Years of School," The Review of Economics and Statistics, MIT Press, vol. 86(2), pages 447-464, May.
    50. Gary Solon & Steven J. Haider & Jeffrey Wooldridge, 2013. "What Are We Weighting For?," NBER Working Papers 18859, National Bureau of Economic Research, Inc.
    51. repec:hrv:faseco:33077826 is not listed on IDEAS
    52. William Rhodes, 2010. "Heterogeneous Treatment Effects: What Does a Regression Estimate?," Evaluation Review, , vol. 34(4), pages 334-361, August.
    53. Dan A. Black & Jeffrey A. Smith & Mark C. Berger & Brett J. Noel, 2003. "Is the Threat of Reemployment Services More Effective Than the Services Themselves? Evidence from Random Assignment in the UI System," American Economic Review, American Economic Association, vol. 93(4), pages 1313-1327, September.
    54. Yitzhaki, Shlomo, 1996. "On Using Linear Regressions in Welfare Economics," Journal of Business & Economic Statistics, American Statistical Association, vol. 14(4), pages 478-486, October.
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Sloczynski, Tymon, 2013. "Population Average Gender Effects," IZA Discussion Papers 7315, Institute of Labor Economics (IZA).
    2. Boris Kaiser, 2013. "Decomposing Differences in Arithmetic Means: A Doubly-Robust Estimation Approach," Diskussionsschriften dp1308, Universitaet Bern, Departement Volkswirtschaft.
    3. Marenya, Paswel & Kassie, Menale & Jaleta, Moti & Rahut, Dil Bahadur, 2015. "Does gender of the household head explain smallholder farmers' maize market positions? Evidence from Ethiopia," 2015 Conference, August 9-14, 2015, Milan, Italy 212229, International Association of Agricultural Economists.
    4. Himaz, Rozana, 2020. "Sweet are the fruit of adversity? The impact of fathers’ death on child non-cognitive outcomes in Ethiopia," Economics & Human Biology, Elsevier, vol. 38(C).
    5. Boris Kaiser, 2016. "Decomposing differences in arithmetic means: a doubly robust estimation approach," Empirical Economics, Springer, vol. 50(3), pages 873-899, May.

    More about this item


    heterogeneity; linear regression; ordinary least squares; propensity score; treatment effects;

    JEL classification:

    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • D72 - Microeconomics - - Analysis of Collective Decision-Making - - - Political Processes: Rent-seeking, Lobbying, Elections, Legislatures, and Voting Behavior
    • F14 - International Economics - - Trade - - - Empirical Studies of Trade
    • O17 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Formal and Informal Sectors; Shadow Economy; Institutional Arrangements

    NEP fields

    This paper has been announced in the following NEP Reports:


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp9491. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Holger Hinte). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.