IDEAS home Printed from https://ideas.repec.org/a/bpj/causin/v9y2021i1p1-8n1.html
   My bibliography  Save this article

Two seemingly paradoxical results in linear models: the variance inflation factor and the analysis of covariance

Author

Listed:
  • Ding Peng

    (Department of Statistics, University of California, Berkeley, CA94720, United States of America)

Abstract

A result from a standard linear model course is that the variance of the ordinary least squares (OLS) coefficient of a variable will never decrease when including additional covariates into the regression. The variance inflation factor (VIF) measures the increase of the variance. Another result from a standard linear model or experimental design course is that including additional covariates in a linear model of the outcome on the treatment indicator will never increase the variance of the OLS coefficient of the treatment at least asymptotically. This technique is called the analysis of covariance (ANCOVA), which is often used to improve the efficiency of treatment effect estimation. So we have two paradoxical results: adding covariates never decreases the variance in the first result but never increases the variance in the second result. In fact, these two results are derived under different assumptions. More precisely, the VIF result conditions on the treatment indicators but the ANCOVA result averages over them. Comparing the estimators with and without adjusting for additional covariates in a completely randomized experiment, I show that the former has smaller variance averaging over the treatment indicators, and the latter has smaller variance at the cost of a larger bias conditioning on the treatment indicators. Therefore, there is no real paradox.

Suggested Citation

  • Ding Peng, 2021. "Two seemingly paradoxical results in linear models: the variance inflation factor and the analysis of covariance," Journal of Causal Inference, De Gruyter, vol. 9(1), pages 1-8, January.
    • Handle: RePEc:bpj:causin:v:9:y:2021:i:1:p:1-8:n:1
    DOI: 10.1515/jci-2019-0023
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/jci-2019-0023
    Download Restriction: no
    File URL: https://libkey.io/10.1515/jci-2019-0023?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Xinran Li & Peng Ding, 2017. "General Forms of Finite Population Central Limit Theorems with Applications to Causal Inference," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(520), pages 1759-1769, October.
    2. Ding, Peng, 2021. "The Frisch–Waugh–Lovell theorem for standard errors," Statistics & Probability Letters, Elsevier, vol. 168(C).
    3. Xinran Li & Peng Ding, 2020. "Rerandomization and regression adjustment," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 82(1), pages 241-268, February.
    4. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Clément de Chaisemartin, 2022. "Trading-off Bias and Variance in Stratified Experiments and in Staggered Adoption Designs, Under a Boundedness Condition on the Magnitude of the Treatment Effect," Working Papers hal-03873919, HAL.
    2. Peter Z. Schochet, 2018. "Design-Based Estimators for Average Treatment Effects for Multi-Armed RCTs," Journal of Educational and Behavioral Statistics, , vol. 43(5), pages 568-593, October.
    3. Iavor Bojinov & David Simchi-Levi & Jinglong Zhao, 2023. "Design and Analysis of Switchback Experiments," Management Science, INFORMS, vol. 69(7), pages 3759-3777, July.
    4. Jiafeng Chen, 2021. "Nonparametric Treatment Effect Identification in School Choice," Papers 2112.03872, arXiv.org, revised Oct 2023.
    5. Peter Z. Schochet, 2020. "Analyzing Grouped Administrative Data for RCTs Using Design-Based Methods," Journal of Educational and Behavioral Statistics, , vol. 45(1), pages 32-57, February.
    6. Haoge Chang, 2023. "Design-based Estimation Theory for Complex Experiments," Papers 2311.06891, arXiv.org.
    7. Iavor Bojinov & Ashesh Rambachan & Neil Shephard, 2021. "Panel experiments and dynamic causal effects: A finite population perspective," Quantitative Economics, Econometric Society, vol. 12(4), pages 1171-1196, November.
    8. Peter L. Cohen & Colin B. Fogarty, 2022. "Gaussian prepivoting for finite population causal inference," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(2), pages 295-320, April.
    9. Edward Wu & Johann A. Gagnon-Bartsch, 2021. "Design-Based Covariate Adjustments in Paired Experiments," Journal of Educational and Behavioral Statistics, , vol. 46(1), pages 109-132, February.
    10. Ke Zhu & Hanzhong Liu, 2023. "Pair‐switching rerandomization," Biometrics, The International Biometric Society, vol. 79(3), pages 2127-2142, September.
    11. Lupparelli, Monia & Mattei, Alessandra, 2020. "Joint and marginal causal effects for binary non-independent outcomes," Journal of Multivariate Analysis, Elsevier, vol. 178(C).
    12. Yuchen Hu & Stefan Wager, 2022. "Switchback Experiments under Geometric Mixing," Papers 2209.00197, arXiv.org, revised Apr 2024.
    13. Harold D Chiang & Yukitoshi Matsushita & Taisuke Otsu, 2023. "Regression adjustment in randomized controlled trials with many covariates," STICERD - Econometrics Paper Series 627, Suntory and Toyota International Centres for Economics and Related Disciplines, LSE.
    14. Fangzhou Su & Peng Ding, 2021. "Model‐assisted analyses of cluster‐randomized experiments," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(5), pages 994-1015, November.
    15. Zhao, Anqi & Ding, Peng, 2021. "Covariate-adjusted Fisher randomization tests for the average treatment effect," Journal of Econometrics, Elsevier, vol. 225(2), pages 278-294.
    16. Evan T.R. Rosenman & Guillaume Basse & Art B. Owen & Mike Baiocchi, 2023. "Combining observational and experimental datasets using shrinkage estimators," Biometrics, The International Biometric Society, vol. 79(4), pages 2961-2973, December.
    17. Zach Branson & Tirthankar Dasgupta, 2020. "Sampling‐based Randomised Designs for Causal Inference under the Potential Outcomes Framework," International Statistical Review, International Statistical Institute, vol. 88(1), pages 101-121, April.
    18. Purevdorj Tuvaandorj, 2024. "A Combinatorial Central Limit Theorem for Stratified Randomization," Papers 2402.14764, arXiv.org, revised Apr 2024.
    19. Harold D Chiang & Yukitoshi Matsushita & Taisuke Otsu, 2023. "Regression adjustment in randomized controlled trials with many covariates," Papers 2302.00469, arXiv.org, revised Nov 2023.
    20. Antoine Deeb & Cl'ement de Chaisemartin, 2019. "Clustering and External Validity in Randomized Controlled Trials," Papers 1912.01052, arXiv.org, revised Dec 2022.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:causin:v:9:y:2021:i:1:p:1-8:n:1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyter.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.