IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/30108.html
   My bibliography  Save this paper

Contamination Bias in Linear Regressions

Author

Listed:
  • Paul Goldsmith-Pinkham
  • Peter Hull
  • Michal Kolesár

Abstract

We study regressions with multiple treatments and a set of controls that is flexible enough to purge omitted variable bias. We show that these regressions generally fail to estimate convex averages of heterogeneous treatment effects—instead, estimates of each treatment’s effect are contaminated by non-convex averages of the effects of other treatments. We discuss three estimation approaches that avoid such contamination bias, including the targeting of easiest-to-estimate weighted average effects. A re-analysis of nine empirical applications finds economically and statistically meaningful contamination bias in observational studies; contamination bias in experimental studies is more limited due to smaller variability in propensity scores.

Suggested Citation

  • Paul Goldsmith-Pinkham & Peter Hull & Michal Kolesár, 2022. "Contamination Bias in Linear Regressions," NBER Working Papers 30108, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:30108
    Note: DEV ED LS TWP
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w30108.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2009. "Dealing with limited overlap in estimation of average treatment effects," Biometrika, Biometrika Trust, vol. 96(1), pages 187-199.
    2. Will Dobbie & Jae Song, 2015. "Debt Relief and Debtor Outcomes: Measuring the Effects of Consumer Bankruptcy Protection," American Economic Review, American Economic Association, vol. 105(3), pages 1272-1311, March.
    3. Luc Behaghel & Bruno Crépon & Marc Gurgand, 2013. "Robustness of the encouragement design in a two-treatment randomized control trial," Working Papers halshs-00834169, HAL.
    4. Alan B. Krueger, 1999. "Experimental Estimates of Education Production Functions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(2), pages 497-532.
    5. David Arnold & Will Dobbie & Peter Hull, 2022. "Measuring Racial Discrimination in Bail Decisions," American Economic Review, American Economic Association, vol. 112(9), pages 2992-3038, September.
    6. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    7. Clément de Chaisemartin & Xavier D'Haultfœuille, 2020. "Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects," American Economic Review, American Economic Association, vol. 110(9), pages 2964-2996, September.
    8. Angrist, Joshua D. & Krueger, Alan B., 1999. "Empirical strategies in labor economics," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 23, pages 1277-1366, Elsevier.
    9. Thomas J. Kane & Douglas O. Staiger, 2008. "Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation," NBER Working Papers 14607, National Bureau of Economic Research, Inc.
    10. Cattaneo, Matias D., 2010. "Efficient semiparametric estimation of multi-valued treatment effects under ignorability," Journal of Econometrics, Elsevier, vol. 155(2), pages 138-154, April.
    11. Patrick Kline & Christopher R. Walters, 2016. "Evaluating Public Programs with Close Substitutes: The Case of HeadStart," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(4), pages 1795-1848.
    12. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    13. Peter Hull, 2018. "Estimating Treatment Effects in Mover Designs," Papers 1804.06721, arXiv.org.
    14. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    15. Tymon Słoczyński, 2022. "Interpreting OLS Estimands When Treatment Effects Are Heterogeneous: Smaller Groups Get Larger Weights," The Review of Economics and Statistics, MIT Press, vol. 104(3), pages 501-509, May.
    16. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    17. Nicole Maestas & Kathleen J. Mullen & Alexander Strand, 2013. "Does Disability Insurance Receipt Discourage Work? Using Examiner Assignment to Estimate Causal Effects of SSDI Receipt," American Economic Review, American Economic Association, vol. 103(5), pages 1797-1829, August.
    18. Jason Abaluck & Mauricio Caceres Bravo & Peter Hull: & Amanda Starc, 2021. "Mortality Effects and Choice Across Private Health Insurance Plans," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 136(3), pages 1557-1610.
    19. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    20. Newey, Whitney K, 1994. "The Asymptotic Variance of Semiparametric Estimators," Econometrica, Econometric Society, vol. 62(6), pages 1349-1382, November.
    21. Matias D. Cattaneo, 2010. "multi-valued treatment effects," The New Palgrave Dictionary of Economics,, Palgrave Macmillan.
    22. Patrick Kline & Christopher R. Walters, 2019. "On Heckits, LATE, and Numerical Equivalence," Econometrica, Econometric Society, vol. 87(2), pages 677-696, March.
    23. Victor Chernozhukov & Whitney K. Newey & Victor Quintas-Martinez & Vasilis Syrgkanis, 2021. "Automatic Debiased Machine Learning via Riesz Regression," Papers 2104.14737, arXiv.org, revised Mar 2024.
    24. Jeffrey R. Kling, 2006. "Incarceration Length, Employment, and Earnings," American Economic Review, American Economic Association, vol. 96(3), pages 863-876, June.
    25. Joshua D. Angrist & Peter D. Hull & Parag A. Pathak & Christopher R. Walters, 2017. "Leveraging Lotteries for School Value-Added: Testing and Estimation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(2), pages 871-919.
    26. Skinner, Jonathan, 2011. "Causes and Consequences of Regional Variations in Health Care," Handbook of Health Economics, in: Mark V. Pauly & Thomas G. Mcguire & Pedro P. Barros (ed.), Handbook of Health Economics, volume 2, chapter 0, pages 45-93, Elsevier.
    27. Joshua D. Angrist, 1998. "Estimating the Labor Market Impact of Voluntary Military Service Using Social Security Data on Military Applicants," Econometrica, Econometric Society, vol. 66(2), pages 249-288, March.
    28. Callaway, Brantly & Sant’Anna, Pedro H.C., 2021. "Difference-in-Differences with multiple time periods," Journal of Econometrics, Elsevier, vol. 225(2), pages 200-230.
    29. Christine Blandhol & John Bonney & Magne Mogstad & Alexander Torgovitsky, 2022. "When is TSLS Actually LATE?," NBER Working Papers 29709, National Bureau of Economic Research, Inc.
    30. Victor Chernozhukov & Whitney K. Newey & Rahul Singh, 2022. "Automatic Debiased Machine Learning of Causal and Structural Effects," Econometrica, Econometric Society, vol. 90(3), pages 967-1027, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Goller, Daniel & Diem, Andrea & Wolter, Stefan C., 2023. "Sitting next to a dropout: Academic success of students with more educated peers," Economics of Education Review, Elsevier, vol. 93(C).
    2. Card, David & Colella, Fabrizio & Lalive, Rafael, 2021. "Gender Preferences in Job Vacancies and Workplace Gender Diversity," IZA Discussion Papers 14758, Institute of Labor Economics (IZA).
    3. Kirill Borusyak & Peter Hull & Xavier Jaravel, 2023. "Design-Based Identification with Formula Instruments: A Review," NBER Working Papers 31393, National Bureau of Economic Research, Inc.
    4. Sam Sims & Harry Fletcher-Wood & Thomas Godfrey-Faussett & Peps Mccrea & Stefanie Meliss, 2023. "Modelling evidence-based practice in initial teacher training: causal effects on teachers' skills, knowledge and self-efficacy," CEPEO Working Paper Series 23-09, UCL Centre for Education Policy and Equalising Opportunities, revised Aug 2023.
    5. Ketel, Nadine & Oosterbeek, Hessel & Sóvágó, Sándor & van der Klaauw, Bas, 2023. "The (un)importance of school assignment," CEPR Discussion Papers 18586, C.E.P.R. Discussion Papers.
    6. Goussé, Marion & Leturcq, Marion, 2022. "More or less unmarried. The impact of legal settings of cohabitation on labour market outcomes," European Economic Review, Elsevier, vol. 149(C).
    7. Gabriel Okasa & Kenneth A. Younge, 2022. "Sample Fit Reliability," Papers 2209.06631, arXiv.org.
    8. Peter Hull & Michal Kolesár & Christopher Walters, 2022. "Labor by design: contributions of David Card, Joshua Angrist, and Guido Imbens," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(3), pages 603-645, July.
    9. Leonard Goff, 2022. "Causal identification with subjective outcomes," Papers 2212.14622, arXiv.org, revised Feb 2023.
    10. Alvarez, Luis A.F. & Toneto, Rodrigo, 2024. "The interpretation of 2SLS with a continuous instrument: A weighted LATE representation," Economics Letters, Elsevier, vol. 237(C).
    11. Francesco Ruggieri, 2023. "Dynamic Regression Discontinuity: A Within-Design Approach," Papers 2307.14203, arXiv.org.
    12. Nibbering, Didier & Oosterveen, Matthijs & Silva, Pedro Luís, 2022. "Clustered Local Average Treatment Effects: Fields of Study and Academic Student Progress," IZA Discussion Papers 15159, Institute of Labor Economics (IZA).
    13. Federico A. Bugni & Ivan A. Canay & Steve McBride, 2023. "Decomposition and Interpretation of Treatment Effects in Settings with Delayed Outcomes," Papers 2302.11505, arXiv.org, revised Sep 2024.
    14. Daniel Goller & Andrea Diem & Stefan C. Wolter, 2022. "Sitting next to a dropout: Study success of students with peers that came to the lecture hall by a different route," Economics of Education Working Paper Series 0190, University of Zurich, Department of Business Administration (IBW).
    15. Winkelmann Rainer, 2024. "Neglected Heterogeneity, Simpson’s Paradox, and the Anatomy of Least Squares," Journal of Econometric Methods, De Gruyter, vol. 13(1), pages 131-144, January.
    16. Jean-Baptiste Bonnier, 2024. "A Split-Treatment Design," Working Papers 2024-11, CRESE.
    17. McManus, Emma & Richardson, Joseph & Wattal, Vasudha & Woodard, Ritchie, 2023. "A Replication of "When a Doctor Falls from the Sky: The Impact of Easing Doctor Supply Constraints on Mortality", Okeke E.N. (2023)," I4R Discussion Paper Series 53, The Institute for Replication (I4R).
    18. Jiafeng Chen, 2021. "Nonparametric Treatment Effect Identification in School Choice," Papers 2112.03872, arXiv.org, revised Oct 2023.
    19. Bernardus F Nazar Van Doornik & Armando Gomes & David Schoenherr & Janis Skrastins, 2023. "Financial access and labor market outcomes: evidence from credit lotteries," BIS Working Papers 1071, Bank for International Settlements.
    20. Rainer Winkelmann, 2023. "Neglected heterogeneity, Simpson’s paradox, and the anatomy of least squares," ECON - Working Papers 426, Department of Economics - University of Zurich, revised Jul 2023.
    21. Ketel, Nadine & Oosterbeek, Hessel & Sóvágó, Sándor & van der Klaauw, Bas, 2023. "The (un)importance of school assignment," CEPR Discussion Papers 18586, C.E.P.R. Discussion Papers.
    22. Michael P. Leung & Pantelis Loupos, 2022. "Graph Neural Networks for Causal Inference Under Network Confounding," Papers 2211.07823, arXiv.org, revised Mar 2024.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Paul Goldsmith-Pinkham & Peter Hull & Michal Kolesár, 2021. "On Estimating Multiple Treatment Effects with Regression," Working Papers 2021-41, Princeton University. Economics Department..
    2. Sloczynski, Tymon, 2018. "A General Weighted Average Representation of the Ordinary and Two-Stage Least Squares Estimands," IZA Discussion Papers 11866, Institute of Labor Economics (IZA).
    3. Tymon S{l}oczy'nski, 2018. "Interpreting OLS Estimands When Treatment Effects Are Heterogeneous: Smaller Groups Get Larger Weights," Papers 1810.01576, arXiv.org, revised May 2020.
    4. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    5. Huber, Martin, 2019. "An introduction to flexible methods for policy evaluation," FSES Working Papers 504, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    6. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    7. Peter Hull & Michal Kolesár & Christopher Walters, 2022. "Labor by design: contributions of David Card, Joshua Angrist, and Guido Imbens," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(3), pages 603-645, July.
    8. Pedro H. C. Sant'Anna, 2016. "Program Evaluation with Right-Censored Data," Papers 1604.02642, arXiv.org.
    9. Committee, Nobel Prize, 2021. "Answering causal questions using observational data," Nobel Prize in Economics documents 2021-2, Nobel Prize Committee.
    10. Ivan A Canay & Magne Mogstad & Jack Mount, 2024. "On the Use of Outcome Tests for Detecting Bias in Decision Making," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 91(4), pages 2135-2167.
    11. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    12. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    13. Tamara Bischof & Boris Kaiser, 2021. "Who cares when you close down? The effects of primary care practice closures on patients," Health Economics, John Wiley & Sons, Ltd., vol. 30(9), pages 2004-2025, September.
    14. Goodman-Bacon, Andrew, 2021. "Difference-in-differences with variation in treatment timing," Journal of Econometrics, Elsevier, vol. 225(2), pages 254-277.
    15. Roberto Esposti, 2022. "The Coevolution of Policy Support and Farmers' Behaviour. An investigation on Italian agriculture over the 2008-2019 period," Working Papers 464, Universita' Politecnica delle Marche (I), Dipartimento di Scienze Economiche e Sociali.
    16. Bär, Marlies & Bakx, Pieter & Wouterse, Bram & van Doorslaer, Eddy, 2022. "Estimating the health value added by nursing homes," Journal of Economic Behavior & Organization, Elsevier, vol. 203(C), pages 1-23.
    17. Samuel Bazzi & Sudarno Sumarto & Asep Suryahadi, "undated". "It’s All in the Timing: Household Expenditure and Labor Supply Responses to Unconditional Cash Transfers," Working Papers 280, Publications Department.
    18. Ivan A Canay & Magne Mogstad & Jack Mount, 2024. "On the Use of Outcome Tests for Detecting Bias in Decision Making," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 91(4), pages 2135-2167.
    19. Bhuller, Manudeep & Sigstad, Henrik, 2024. "2SLS with multiple treatments," Journal of Econometrics, Elsevier, vol. 242(1).
    20. Jesse Rothstein & Till von Wachter, 2016. "Social Experiments in the Labor Market," NBER Working Papers 22585, National Bureau of Economic Research, Inc.

    More about this item

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:30108. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.