IDEAS home Printed from https://ideas.repec.org/a/bla/biomet/v77y2021i4p1276-1288.html
   My bibliography  Save this article

Evaluating and improving a matched comparison of antidepressants and bone density

Author

Listed:
  • Ruoqi Yu

Abstract

Matching is a common approach to covariate adjustment in estimating causal effects in observational studies. It is important to assess covariate balance of the matched samples. This is usually done informally, in ways that have a number of limitations. First, there are many diagnostics, even if covariates are assessed one at a time, which raises multiplicity issues. In addition, joint distributions of covariates, even bivariate distributions, are often ignored. Further, it is an open question whether diagnostics identify the major problems. To address these issues, a formal assessment of covariate balance is developed in the current paper. Unlike the common informal diagnostics, the proposed method compares both marginal distributions and joint distributions of the matched sample with those of the benchmark, complete randomizations. The method controls the probability of falsely identifying a covariate imbalance among many comparisons, yet it has a high probability of correctly detecting and identifying a major problem. An R package met implementing the proposed method is available on CRAN.

Suggested Citation

  • Ruoqi Yu, 2021. "Evaluating and improving a matched comparison of antidepressants and bone density," Biometrics, The International Biometric Society, vol. 77(4), pages 1276-1288, December.
    • Handle: RePEc:bla:biomet:v:77:y:2021:i:4:p:1276-1288
    DOI: 10.1111/biom.13374
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/biom.13374
    Download Restriction: no
    File URL: https://libkey.io/10.1111/biom.13374?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    2. Jinyong Hahn, 2004. "Functional Restriction and Efficiency in Causal Inference," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 73-76, February.
    3. Paul R. Rosenbaum, 2005. "An exact distribution‐free test comparing two multivariate distributions based on adjacency," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(4), pages 515-530, September.
    4. Samuel D. Pimentel & Dylan S. Small & Paul R. Rosenbaum, 2016. "Constructed Second Control Groups and Attenuation of Unmeasured Biases," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(515), pages 1157-1167, July.
    5. Paul R. Rosenbaum, 2007. "Sensitivity Analysis for m-Estimates, Tests, and Confidence Intervals in Matched Observational Studies," Biometrics, The International Biometric Society, vol. 63(2), pages 456-464, June.
    6. Colin B. Fogarty & Mark E. Mikkelsen & David F. Gaieski & Dylan S. Small, 2016. "Discrete Optimization for Interpretable Study Populations and Randomization Inference in an Observational Study of Severe Sepsis Mortality," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(514), pages 447-458, April.
    7. Paul R. Rosenbaum, 2015. "Bahadur Efficiency of Sensitivity Analyses in Observational Studies," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(509), pages 205-217, March.
    8. Dan Yang & Dylan S. Small & Jeffrey H. Silber & Paul R. Rosenbaum, 2012. "Optimal Matching with Minimal Deviation from Fine Balance in a Study of Obesity and Surgical Outcomes," Biometrics, The International Biometric Society, vol. 68(2), pages 628-636, June.
    9. Kosuke Imai & Marc Ratkovic, 2014. "Covariate balancing propensity score," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 76(1), pages 243-263, January.
    10. José R. Zubizarreta, 2015. "Stable Weights that Balance Covariates for Estimation With Incomplete Outcome Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(511), pages 910-922, September.
    11. Kwun Chuen Gary Chan & Sheung Chi Phillip Yam & Zheng Zhang, 2016. "Globally efficient non-parametric inference of average treatment effects by empirical balancing calibration weighting," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(3), pages 673-700, June.
    12. Ruoqi Yu & Paul R. Rosenbaum, 2019. "Directional penalties for optimal matching in observational studies," Biometrics, The International Biometric Society, vol. 75(4), pages 1380-1390, December.
    13. Rosenbaum, Paul R., 2010. "Design Sensitivity and Efficiency in Observational Studies," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 692-702.
    14. Paul R. Rosenbaum, 2008. "Testing hypotheses in order," Biometrika, Biometrika Trust, vol. 95(1), pages 248-252.
    15. Yixin Wang & Jose R Zubizarreta, 2020. "Minimal dispersion approximately balancing weights: asymptotic properties and practical considerations," Biometrika, Biometrika Trust, vol. 107(1), pages 93-105.
    16. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    17. Samuel D. Pimentel & Rachel R. Kelz & Jeffrey H. Silber & Paul R. Rosenbaum, 2015. "Large, Sparse Optimal Matching With Refined Covariate Balance in an Observational Study of the Health Outcomes Produced by New Surgeons," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(510), pages 515-527, June.
    18. José R. Zubizarreta, 2012. "Using Mixed Integer Programming for Matching in an Observational Study of Kidney Failure After Surgery," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(500), pages 1360-1371, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bo Zhang, 2023. "Efficient algorithms for building representative matched pairs with enhanced generalizability," Biometrics, The International Biometric Society, vol. 79(4), pages 3981-3997, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. María de los Angeles Resa & José R. Zubizarreta, 2020. "Direct and stable weight adjustment in non‐experimental studies with multivalued treatments: analysis of the effect of an earthquake on post‐traumatic stress," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(4), pages 1387-1410, October.
    2. Pedro H. C. Sant'Anna & Xiaojun Song & Qi Xu, 2022. "Covariate distribution balance via propensity scores," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(6), pages 1093-1120, September.
    3. Shixiao Zhang & Peisong Han & Changbao Wu, 2023. "Calibration Techniques Encompassing Survey Sampling, Missing Data Analysis and Causal Inference," International Statistical Review, International Statistical Institute, vol. 91(2), pages 165-192, August.
    4. Jason J. Sauppe & Sheldon H. Jacobson, 2017. "The role of covariate balance in observational studies," Naval Research Logistics (NRL), John Wiley & Sons, vol. 64(4), pages 323-344, June.
    5. Bikram Karmakar, 2022. "An approximation algorithm for blocking of an experimental design," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(5), pages 1726-1750, November.
    6. Cousineau, Martin & Verter, Vedat & Murphy, Susan A. & Pineau, Joelle, 2023. "Estimating causal effects with optimization-based methods: A review and empirical comparison," European Journal of Operational Research, Elsevier, vol. 304(2), pages 367-380.
    7. Ganesh Karapakula, 2023. "Stable Probability Weighting: Large-Sample and Finite-Sample Estimation and Inference Methods for Heterogeneous Causal Effects of Multivalued Treatments Under Limited Overlap," Papers 2301.05703, arXiv.org, revised Jan 2023.
    8. Vahe Avagyan & Stijn Vansteelandt, 2021. "Stable inverse probability weighting estimation for longitudinal studies," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 48(3), pages 1046-1067, September.
    9. Ruoqi Yu, 2023. "How well can fine balance work for covariate balancing," Biometrics, The International Biometric Society, vol. 79(3), pages 2346-2356, September.
    10. Brett R. Gordon & Florian Zettelmeyer & Neha Bhargava & Dan Chapsky, 2019. "A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook," Marketing Science, INFORMS, vol. 38(2), pages 193-225, March.
    11. Phillip Heiler, 2020. "Efficient Covariate Balancing for the Local Average Treatment Effect," Papers 2007.04346, arXiv.org.
    12. Siyu Heng & Hyunseung Kang & Dylan S. Small & Colin B. Fogarty, 2021. "Increasing power for observational studies of aberrant response: An adaptive approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(3), pages 482-504, July.
    13. Sean Yiu & Li Su, 2022. "Joint calibrated estimation of inverse probability of treatment and censoring weights for marginal structural models," Biometrics, The International Biometric Society, vol. 78(1), pages 115-127, March.
    14. Chunrong Ai & Lukang Huang & Zheng Zhang, 2018. "A Simple and Efficient Estimation of the Average Treatment Effect in the Presence of Unmeasured Confounders," Papers 1807.05678, arXiv.org.
    15. Martin Cousineau & Vedat Verter & Susan A. Murphy & Joelle Pineau, 2022. "Estimating causal effects with optimization-based methods: A review and empirical comparison," Papers 2203.00097, arXiv.org.
    16. Susan Athey & Guido W. Imbens & Stefan Wager, 2018. "Approximate residual balancing: debiased inference of average treatment effects in high dimensions," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 80(4), pages 597-623, September.
    17. Hirschauer, Norbert & Grüner, Sven & Mußhoff, Oliver & Becker, Claudia & Jantsch, Antje, 2020. "Can p-values be meaningfully interpreted without random sampling?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 14, pages 71-91.
    18. Dasom Lee & Shu Yang & Lin Dong & Xiaofei Wang & Donglin Zeng & Jianwen Cai, 2023. "Improving trial generalizability using observational studies," Biometrics, The International Biometric Society, vol. 79(2), pages 1213-1225, June.
    19. Ian Gazeley & Rose Holmes & Andrew Newell & Kevin Reynolds & Hector Gutierrez Rufrancos, 2023. "Escaping from hunger before WW1: the nutritional transition and living standards in Western Europe and USA in the late nineteenth century," Cliometrica, Springer;Cliometric Society (Association Francaise de Cliométrie), vol. 17(3), pages 533-565, September.
    20. Siyu Heng & Dylan S. Small & Paul R. Rosenbaum, 2020. "Finding the strength in a weak instrument in a study of cognitive outcomes produced by Catholic high schools," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(3), pages 935-958, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:biomet:v:77:y:2021:i:4:p:1276-1288. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0006-341X .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.