IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2106.05024.html
   My bibliography  Save this paper

Contamination Bias in Linear Regressions

Author

Listed:
  • Paul Goldsmith-Pinkham
  • Peter Hull
  • Michal Koles'ar

Abstract

We study regressions with multiple treatments and a set of controls that is flexible enough to purge omitted variable bias. We show that these regressions generally fail to estimate convex averages of heterogeneous treatment effects -- instead, estimates of each treatment's effect are contaminated by non-convex averages of the effects of other treatments. We discuss three estimation approaches that avoid such contamination bias, including the targeting of easiest-to-estimate weighted average effects. A re-analysis of nine empirical applications finds economically and statistically meaningful contamination bias in observational studies; contamination bias in experimental studies is more limited due to smaller variability in propensity scores.

Suggested Citation

  • Paul Goldsmith-Pinkham & Peter Hull & Michal Koles'ar, 2021. "Contamination Bias in Linear Regressions," Papers 2106.05024, arXiv.org, revised Jun 2024.
  • Handle: RePEc:arx:papers:2106.05024
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2106.05024
    File Function: Latest version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2009. "Dealing with limited overlap in estimation of average treatment effects," Biometrika, Biometrika Trust, vol. 96(1), pages 187-199.
    2. Will Dobbie & Jae Song, 2015. "Debt Relief and Debtor Outcomes: Measuring the Effects of Consumer Bankruptcy Protection," American Economic Review, American Economic Association, vol. 105(3), pages 1272-1311, March.
    3. David Arnold & Will Dobbie & Peter Hull, 2022. "Measuring Racial Discrimination in Bail Decisions," American Economic Review, American Economic Association, vol. 112(9), pages 2992-3038, September.
    4. Jeffrey R. Kling, 2006. "Incarceration Length, Employment, and Earnings," American Economic Review, American Economic Association, vol. 96(3), pages 863-876, June.
    5. Luc Behaghel & Bruno Crépon & Marc Gurgand, 2013. "Robustness of the encouragement design in a two-treatment randomized control trial," Working Papers halshs-00834169, HAL.
    6. Alan B. Krueger, 1999. "Experimental Estimates of Education Production Functions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(2), pages 497-532.
    7. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    8. Clément de Chaisemartin & Xavier D'Haultfœuille, 2020. "Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects," American Economic Review, American Economic Association, vol. 110(9), pages 2964-2996, September.
    9. Patrick Kline & Christopher R. Walters, 2016. "Evaluating Public Programs with Close Substitutes: The Case of HeadStart," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(4), pages 1795-1848.
    10. Angrist, Joshua D. & Krueger, Alan B., 1999. "Empirical strategies in labor economics," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 23, pages 1277-1366, Elsevier.
    11. Thomas J. Kane & Douglas O. Staiger, 2008. "Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation," NBER Working Papers 14607, National Bureau of Economic Research, Inc.
    12. Tymon Słoczyński, 2022. "Interpreting OLS Estimands When Treatment Effects Are Heterogeneous: Smaller Groups Get Larger Weights," The Review of Economics and Statistics, MIT Press, vol. 104(3), pages 501-509, May.
    13. Joshua D. Angrist & Peter D. Hull & Parag A. Pathak & Christopher R. Walters, 2017. "Leveraging Lotteries for School Value-Added: Testing and Estimation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(2), pages 871-919.
    14. Cattaneo, Matias D., 2010. "Efficient semiparametric estimation of multi-valued treatment effects under ignorability," Journal of Econometrics, Elsevier, vol. 155(2), pages 138-154, April.
    15. Newey, Whitney K, 1994. "The Asymptotic Variance of Semiparametric Estimators," Econometrica, Econometric Society, vol. 62(6), pages 1349-1382, November.
    16. Jason Abaluck & Mauricio Caceres Bravo & Peter Hull: & Amanda Starc, 2021. "Mortality Effects and Choice Across Private Health Insurance Plans," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 136(3), pages 1557-1610.
    17. Patrick Kline & Christopher R. Walters, 2019. "On Heckits, LATE, and Numerical Equivalence," Econometrica, Econometric Society, vol. 87(2), pages 677-696, March.
    18. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    19. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    20. Skinner, Jonathan, 2011. "Causes and Consequences of Regional Variations in Health Care," Handbook of Health Economics, in: Mark V. Pauly & Thomas G. Mcguire & Pedro P. Barros (ed.), Handbook of Health Economics, volume 2, chapter 0, pages 45-93, Elsevier.
    21. Joshua D. Angrist, 1998. "Estimating the Labor Market Impact of Voluntary Military Service Using Social Security Data on Military Applicants," Econometrica, Econometric Society, vol. 66(2), pages 249-288, March.
    22. Callaway, Brantly & Sant’Anna, Pedro H.C., 2021. "Difference-in-Differences with multiple time periods," Journal of Econometrics, Elsevier, vol. 225(2), pages 200-230.
    23. Peter Hull, 2018. "Estimating Treatment Effects in Mover Designs," Papers 1804.06721, arXiv.org.
    24. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    25. Christine Blandhol & John Bonney & Magne Mogstad & Alexander Torgovitsky, 2022. "When is TSLS Actually LATE?," NBER Working Papers 29709, National Bureau of Economic Research, Inc.
    26. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    27. Nicole Maestas & Kathleen J. Mullen & Alexander Strand, 2013. "Does Disability Insurance Receipt Discourage Work? Using Examiner Assignment to Estimate Causal Effects of SSDI Receipt," American Economic Review, American Economic Association, vol. 103(5), pages 1797-1829, August.
    28. Victor Chernozhukov & Whitney K. Newey & Rahul Singh, 2022. "Automatic Debiased Machine Learning of Causal and Structural Effects," Econometrica, Econometric Society, vol. 90(3), pages 967-1027, May.
    29. Matias D. Cattaneo, 2010. "multi-valued treatment effects," The New Palgrave Dictionary of Economics,, Palgrave Macmillan.
    30. Victor Chernozhukov & Whitney K. Newey & Victor Quintas-Martinez & Vasilis Syrgkanis, 2021. "Automatic Debiased Machine Learning via Riesz Regression," Papers 2104.14737, arXiv.org, revised Mar 2024.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jean-Baptiste Bonnier, 2024. "A Split-Treatment Design," Working Papers 2024-11, CRESE.
    2. Goussé, Marion & Leturcq, Marion, 2022. "More or less unmarried. The impact of legal settings of cohabitation on labour market outcomes," European Economic Review, Elsevier, vol. 149(C).
    3. Kirill Borusyak & Peter Hull & Xavier Jaravel, 2023. "Design-based identification with formula instruments: A review," CeMMAP working papers 12/23, Institute for Fiscal Studies.
    4. Goller, Daniel & Diem, Andrea & Wolter, Stefan C., 2023. "Sitting next to a dropout: Academic success of students with more educated peers," Economics of Education Review, Elsevier, vol. 93(C).
    5. Leonard Goff, 2022. "Identifying causal effects with subjective ordinal outcomes," Papers 2212.14622, arXiv.org, revised Dec 2024.
    6. Daniel Goller & Andrea Diem & Stefan C. Wolter, 2022. "Sitting next to a dropout: Study success of students with peers that came to the lecture hall by a different route," Economics of Education Working Paper Series 0190, University of Zurich, Department of Business Administration (IBW).
    7. Alvarez, Luis A.F. & Toneto, Rodrigo, 2024. "The interpretation of 2SLS with a continuous instrument: A weighted LATE representation," Economics Letters, Elsevier, vol. 237(C).
    8. Winkelmann Rainer, 2024. "Neglected Heterogeneity, Simpson’s Paradox, and the Anatomy of Least Squares," Journal of Econometric Methods, De Gruyter, vol. 13(1), pages 131-144, January.
    9. David Card & Fabrizio Colella & Rafael Lalive, 2021. "Gender Preferences in Job Vacancies and Workplace Gender Diversity," NBER Working Papers 29350, National Bureau of Economic Research, Inc.
    10. Jonathan Cohen & Geoffrey C. Schnorr, 2024. "Efficiency Costs of Unemployment Insurance Denial: Evidence from Randomly Assigned Examiners," Upjohn Working Papers 24-404, W.E. Upjohn Institute for Employment Research.
    11. Peter Hull & Michal Kolesár & Christopher Walters, 2022. "Labor by design: contributions of David Card, Joshua Angrist, and Guido Imbens," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(3), pages 603-645, July.
    12. Francesco Ruggieri, 2023. "Dynamic Regression Discontinuity: A Within-Design Approach," Papers 2307.14203, arXiv.org.
    13. Nibbering, Didier & Oosterveen, Matthijs & Silva, Pedro Luís, 2022. "Clustered Local Average Treatment Effects: Fields of Study and Academic Student Progress," IZA Discussion Papers 15159, Institute of Labor Economics (IZA).
    14. Bhuller, Manudeep & Sigstad, Henrik, 2024. "2SLS with multiple treatments," Journal of Econometrics, Elsevier, vol. 242(1).
    15. Sam Sims & Harry Fletcher-Wood & Thomas Godfrey-Faussett & Peps Mccrea & Stefanie Meliss, 2023. "Modelling evidence-based practice in initial teacher training: causal effects on teachers' skills, knowledge and self-efficacy," CEPEO Working Paper Series 23-09, UCL Centre for Education Policy and Equalising Opportunities, revised Aug 2023.
    16. McManus, Emma & Richardson, Joseph & Wattal, Vasudha & Woodard, Ritchie, 2023. "A Replication of "When a Doctor Falls from the Sky: The Impact of Easing Doctor Supply Constraints on Mortality", Okeke E.N. (2023)," I4R Discussion Paper Series 53, The Institute for Replication (I4R).
    17. Federico A. Bugni & Ivan A. Canay & Steve McBride, 2023. "Decomposition and Interpretation of Treatment Effects in Settings with Delayed Outcomes," Papers 2302.11505, arXiv.org, revised Sep 2024.
    18. Ketel, Nadine & Oosterbeek, Hessel & Sovago, Sandor & van der Klaauw, Bas, 2023. "The (Un)Importance of School Assignment," IZA Discussion Papers 16591, Institute of Labor Economics (IZA).
    19. Laura Castell & Marc Gurgand & Clément Imbert & Todor Tochev, 2024. "Take-up of Social Benefits: Experimental Evidence from France," Institut des Politiques Publiques halshs-04720989, HAL.
    20. Jiafeng Chen, 2021. "Nonparametric Treatment Effect Identification in School Choice," Papers 2112.03872, arXiv.org, revised Oct 2023.
    21. Bernardus F Nazar Van Doornik & Armando Gomes & David Schoenherr & Janis Skrastins, 2023. "Financial access and labor market outcomes: evidence from credit lotteries," BIS Working Papers 1071, Bank for International Settlements.
    22. Gabriel Okasa & Kenneth A. Younge, 2022. "Sample Fit Reliability," Papers 2209.06631, arXiv.org.
    23. Rainer Winkelmann, 2023. "Neglected heterogeneity, Simpson’s paradox, and the anatomy of least squares," ECON - Working Papers 426, Department of Economics - University of Zurich, revised Jul 2023.
    24. Michael P. Leung & Pantelis Loupos, 2022. "Graph Neural Networks for Causal Inference Under Network Confounding," Papers 2211.07823, arXiv.org, revised Mar 2024.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Paul Goldsmith-Pinkham & Peter Hull & Michal Kolesár, 2021. "On Estimating Multiple Treatment Effects with Regression," Working Papers 2021-41, Princeton University. Economics Department..
    2. Sloczynski, Tymon, 2018. "A General Weighted Average Representation of the Ordinary and Two-Stage Least Squares Estimands," IZA Discussion Papers 11866, Institute of Labor Economics (IZA).
    3. Tymon S{l}oczy'nski, 2018. "Interpreting OLS Estimands When Treatment Effects Are Heterogeneous: Smaller Groups Get Larger Weights," Papers 1810.01576, arXiv.org, revised May 2020.
    4. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    5. Huber, Martin, 2019. "An introduction to flexible methods for policy evaluation," FSES Working Papers 504, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    6. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    7. Peter Hull & Michal Kolesár & Christopher Walters, 2022. "Labor by design: contributions of David Card, Joshua Angrist, and Guido Imbens," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(3), pages 603-645, July.
    8. Pedro H. C. Sant'Anna, 2016. "Program Evaluation with Right-Censored Data," Papers 1604.02642, arXiv.org.
    9. Bhuller, Manudeep & Sigstad, Henrik, 2024. "2SLS with multiple treatments," Journal of Econometrics, Elsevier, vol. 242(1).
    10. Committee, Nobel Prize, 2021. "Answering causal questions using observational data," Nobel Prize in Economics documents 2021-2, Nobel Prize Committee.
    11. Ivan A Canay & Magne Mogstad & Jack Mount, 2024. "On the Use of Outcome Tests for Detecting Bias in Decision Making," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 91(4), pages 2135-2167.
    12. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    13. Firpo, Sergio Pinheiro & Pinto, Rafael de Carvalho Cayres, 2012. "Combining Strategies for the Estimation of Treatment Effects," Brazilian Review of Econometrics, Sociedade Brasileira de Econometria - SBE, vol. 32(1), March.
    14. Marcus Roller, Daniel Steinberg, 2023. "Differences-in-Differences with multiple Treatments under Control," Diskussionsschriften credresearchpaper41, Universitaet Bern, Departement Volkswirtschaft - CRED.
    15. Jorge Rodríguez & Fernando Saltiel & Sergio Urzúa, 2022. "Dynamic treatment effects of job training," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(2), pages 242-269, March.
    16. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    17. Ganesh Karapakula, 2023. "Stable Probability Weighting: Large-Sample and Finite-Sample Estimation and Inference Methods for Heterogeneous Causal Effects of Multivalued Treatments Under Limited Overlap," Papers 2301.05703, arXiv.org, revised Jan 2023.
    18. Tamara Bischof & Boris Kaiser, 2021. "Who cares when you close down? The effects of primary care practice closures on patients," Health Economics, John Wiley & Sons, Ltd., vol. 30(9), pages 2004-2025, September.
    19. Goodman-Bacon, Andrew, 2021. "Difference-in-differences with variation in treatment timing," Journal of Econometrics, Elsevier, vol. 225(2), pages 254-277.
    20. Rothstein, Jesse & Von Wachter, Till, 2016. "Social Experiments in the Labor Market," Department of Economics, Working Paper Series qt7957p9g6, Department of Economics, Institute for Business and Economic Research, UC Berkeley.

    More about this item

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2106.05024. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.