IDEAS home Printed from https://ideas.repec.org/a/bla/biomet/v79y2023i4p3968-3980.html
   My bibliography  Save this article

A second evidence factor for a second control group

Author

Listed:
  • Paul R. Rosenbaum

Abstract

In an observational study of the effects caused by a treatment, a second control group is used in an effort to detect bias from unmeasured covariates, and the investigator is content if no evidence of bias is found. This strategy is not entirely satisfactory: two control groups may differ significantly, yet the difference may be too small to invalidate inferences about the treatment, or the control groups may not differ yet nonetheless fail to provide a tangible strengthening of the evidence of a treatment effect. Is a firmer conclusion possible? Is there a way to analyze a second control group such that the data might report measurably strengthened evidence of cause and effect, that is, insensitivity to larger unmeasured biases? Evidence factor analyses are not commonly used with a second control group: most analyses compare the treated group to each control group, but analyses of that kind are partially redundant; so, they do not constitute evidence factors. An alternative analysis is proposed here, one that does yield two evidence factors, and with a carefully designed test statistic, is capable of extracting strong evidence from the second factor. The new technical work here concerns the development of a test statistic with high design sensitivity and high Bahadur efficiency in a sensitivity analysis for the second factor. A study of binge drinking as a cause of high blood pressure is used as an illustration.

Suggested Citation

  • Paul R. Rosenbaum, 2023. "A second evidence factor for a second control group," Biometrics, The International Biometric Society, vol. 79(4), pages 3968-3980, December.
    • Handle: RePEc:bla:biomet:v:79:y:2023:i:4:p:3968-3980
    DOI: 10.1111/biom.13921
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/biom.13921
    Download Restriction: no
    File URL: https://libkey.io/10.1111/biom.13921?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Michael J. Daniels & Jason A. Roy & Chanmin Kim & Joseph W. Hogan & Michael G. Perri, 2012. "Bayesian Inference for the Causal Effect of Mediation," Biometrics, The International Biometric Society, vol. 68(4), pages 1028-1036, December.
    2. Jesse Y. Hsu & José R. Zubizarreta & Dylan S. Small & Paul R. Rosenbaum, 2015. "Strong control of the familywise error rate in observational studies that discover effect modification by exploratory methods," Biometrika, Biometrika Trust, vol. 102(4), pages 767-782.
    3. P R Rosenbaum & D B Rubin, 2023. "Propensity scores in the design of observational studies for causal effects," Biometrika, Biometrika Trust, vol. 110(1), pages 1-13.
    4. Jesse Y. Hsu & Dylan S. Small & Paul R. Rosenbaum, 2013. "Effect Modification and Design Sensitivity in Observational Studies," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 108(501), pages 135-148, March.
    5. B Karmakar & B French & D S Small, 2019. "Integrating the evidence from evidence factors in observational studies," Biometrika, Biometrika Trust, vol. 106(2), pages 353-367.
    6. Matteo Bonvini & Edward H. Kennedy, 2022. "Sensitivity Analysis via the Proportion of Unmeasured Confounding," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 117(539), pages 1540-1550, September.
    7. S R Howard & S D Pimentel, 2021. "The uniform general signed rank test and its design sensitivity [A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations]," Biometrika, Biometrika Trust, vol. 108(2), pages 381-396.
    8. Paul R. Rosenbaum, 2007. "Confidence Intervals for Uncommon but Dramatic Responses to Treatment," Biometrics, The International Biometric Society, vol. 63(4), pages 1164-1171, December.
    9. Paul R. Rosenbaum, 2015. "Bahadur Efficiency of Sensitivity Analyses in Observational Studies," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(509), pages 205-217, March.
    10. Rosenbaum, Paul R. & Silber, Jeffrey H., 2009. "Amplification of Sensitivity Analysis in Matched Observational Studies," Journal of the American Statistical Association, American Statistical Association, vol. 104(488), pages 1398-1405.
    11. Paul R. Rosenbaum, 2010. "Evidence factors in observational studies," Biometrika, Biometrika Trust, vol. 97(2), pages 333-345.
    12. Bryan E. Shepherd & Peter B. Gilbert & Yannis Jemiai & Andrea Rotnitzky, 2006. "Sensitivity Analyses Comparing Outcomes Only Existing in a Subset Selected Post-Randomization, Conditional on Covariates, with Application to HIV Vaccine Trials," Biometrics, The International Biometric Society, vol. 62(2), pages 332-342, June.
    13. B. Zhang & D. S. Small & K. B. Lasater & M. McHugh & J. H. Silber & P. R. Rosenbaum, 2023. "Matching One Sample According to Two Criteria in Observational Studies," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 118(542), pages 1140-1151, April.
    14. Colin B. Fogarty & Dylan S. Small, 2016. "Sensitivity Analysis for Multiple Comparisons in Matched Observational Studies Through Quadratically Constrained Linear Programming," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(516), pages 1820-1830, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Siyu Heng & Dylan S. Small & Paul R. Rosenbaum, 2020. "Finding the strength in a weak instrument in a study of cognitive outcomes produced by Catholic high schools," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(3), pages 935-958, June.
    2. Kwonsang Lee & Dylan S. Small & Paul R. Rosenbaum, 2018. "A powerful approach to the study of moderate effect modification in observational studies," Biometrics, The International Biometric Society, vol. 74(4), pages 1161-1170, December.
    3. Raiden B. Hasegawa & Sameer K. Deshpande & Dylan S. Small & Paul R. Rosenbaum, 2020. "Causal Inference With Two Versions of Treatment," Journal of Educational and Behavioral Statistics, , vol. 45(4), pages 426-445, August.
    4. Aditya Ghosh & Dominik Rothenhausler, 2025. "Assumption-robust Causal Inference," Papers 2505.08729, arXiv.org.
    5. Paul R. Rosenbaum, 2023. "Sensitivity analyses informed by tests for bias in observational studies," Biometrics, The International Biometric Society, vol. 79(1), pages 475-487, March.
    6. Paul R. Rosenbaum, 2015. "Bahadur Efficiency of Sensitivity Analyses in Observational Studies," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(509), pages 205-217, March.
    7. Paul R. Rosenbaum & Dylan S. Small, 2017. "An adaptive Mantel–Haenszel test for sensitivity analysis in observational studies," Biometrics, The International Biometric Society, vol. 73(2), pages 422-430, June.
    8. Siyu Heng & Hyunseung Kang & Dylan S. Small & Colin B. Fogarty, 2021. "Increasing power for observational studies of aberrant response: An adaptive approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(3), pages 482-504, July.
    9. Paul R. Rosenbaum, 2011. "A New u-Statistic with Superior Design Sensitivity in Matched Observational Studies," Biometrics, The International Biometric Society, vol. 67(3), pages 1017-1027, September.
    10. Paul R. Rosenbaum, 2015. "Some Counterclaims Undermine Themselves in Observational Studies," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(512), pages 1389-1398, December.
    11. Harrison, Ann E. & Lin, Justin Yifu & Xu, Lixin Colin, 2014. "Explaining Africa’s (Dis)advantage," World Development, Elsevier, vol. 63(C), pages 59-77.
    12. Jawid, Asadullah & Khadjavi, Menusch, 2019. "Adaptation to climate change in Afghanistan: Evidence on the impact of external interventions," Economic Analysis and Policy, Elsevier, vol. 64(C), pages 64-82.
    13. Zelu, Barbara Ama & Iranzo, Susana & Perez-Laborda, Alejandro, 2024. "Financial inclusion and women economic empowerment in Ghana," Emerging Markets Review, Elsevier, vol. 62(C).
    14. Shuxi Zeng & Elizabeth C. Lange & Elizabeth A. Archie & Fernando A. Campos & Susan C. Alberts & Fan Li, 2023. "A Causal Mediation Model for Longitudinal Mediators and Survival Outcomes with an Application to Animal Behavior," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 28(2), pages 197-218, June.
    15. VanderWeele Tyler J, 2011. "Principal Stratification -- Uses and Limitations," The International Journal of Biostatistics, De Gruyter, vol. 7(1), pages 1-14, July.
    16. Matthew A. Masten & Alexandre Poirier, 2018. "Identification of Treatment Effects Under Conditional Partial Independence," Econometrica, Econometric Society, vol. 86(1), pages 317-351, January.
    17. Francesco Bartolucci & Donata Favaro & Fulvia Pennoni & Dario Sciulli, 2024. "An Analysis of the Effect of Streaming on Civic Participation Through a Causal Hidden Markov Model," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 172(1), pages 163-190, March.
    18. Yen-Cheng Chen & Hsiang-Chun Lin, 2020. "Exploring Effective Sensory Experience in the Environmental Design of Sustainable Cafés," IJERPH, MDPI, vol. 17(23), pages 1-16, December.
    19. Linbo Wang & Thomas S. Richardson & Xiao-Hua Zhou, 2017. "Causal analysis of ordinal treatments and binary outcomes under truncation by death," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(3), pages 719-735, June.
    20. Dean Follmann & Michael P. Fay & Michael Proschan, 2009. "Chop-Lump Tests for Vaccine Trials," Biometrics, The International Biometric Society, vol. 65(3), pages 885-893, September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:biomet:v:79:y:2023:i:4:p:3968-3980. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0006-341X .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.