IDEAS home Printed from https://ideas.repec.org/a/bla/biomet/v78y2022i1p352-363.html
   My bibliography  Save this article

A closed max‐t test for multiple comparisons of areas under the ROC curve

Author

Listed:
  • Paul Blanche
  • Jean‐François Dartigues
  • Jérémie Riou

Abstract

Comparing areas under the ROC curve (AUCs) is a popular approach to compare prognostic biomarkers. The aim of this paper is to present an efficient method to control the family‐wise error rate when multiple comparisons are performed. We suggest to combine the max‐t test and the closed testing procedures. We build on previous work on asymptotic results for ROC curves and on general multiple testing methods to efficiently take into account both the correlations between the test statistics and the logical constraints between the null hypotheses. The proposed method results in an uniformly more powerful procedure than both the single‐step max‐t test procedure and popular stepwise extensions of the Bonferroni procedure, such as Bonferroni–Holm. As demonstrated in this paper, the method can be applied in most usual contexts, including the time‐dependent context with right censored data. We show how the method works in practice through a motivating example where we compare several psychometric scores to predict the t‐year risk of Alzheimer's disease. The example illustrates several multiple testing settings and demonstrates the advantage of using the proposed methods over common alternatives. R code has been made available to facilitate the use of the methods by others.

Suggested Citation

  • Paul Blanche & Jean‐François Dartigues & Jérémie Riou, 2022. "A closed max‐t test for multiple comparisons of areas under the ROC curve," Biometrics, The International Biometric Society, vol. 78(1), pages 352-363, March.
    • Handle: RePEc:bla:biomet:v:78:y:2022:i:1:p:352-363
    DOI: 10.1111/biom.13401
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/biom.13401
    Download Restriction: no
    File URL: https://libkey.io/10.1111/biom.13401?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Christian Bressen Pipper & Christian Ritz & Hans Bisgaard, 2012. "A versatile method for confirmatory evaluation of the effects of a covariate in multiple models," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 61(2), pages 315-326, March.
    2. Uno, Hajime & Cai, Tianxi & Tian, Lu & Wei, L.J., 2007. "Evaluating Prediction Rules for t-Year Survivors With Censored Regression Models," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 527-537, June.
    3. Westfall, Peter H. & Tobias, Randall D., 2007. "Multiple Testing of General Contrasts: Truncated Closure and the Extended ShafferRoyen Method," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 487-494, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Brice Ozenne & Esben Budtz-Jørgensen & Sebastian Elgaard Ebert, 2023. "Controlling the familywise error rate when performing multiple comparisons in a linear latent variable model," Computational Statistics, Springer, vol. 38(1), pages 1-23, March.
    2. Yu Zheng & Tianxi Cai, 2017. "Augmented estimation for t‐year survival with censored regression models," Biometrics, The International Biometric Society, vol. 73(4), pages 1169-1178, December.
    3. Alessandro, Martin & Cardinale Lagomarsino, Bruno & Scartascini, Carlos & Streb, Jorge & Torrealday, Jerónimo, 2021. "Transparency and Trust in Government. Evidence from a Survey Experiment," World Development, Elsevier, vol. 138(C).
    4. Paul Frédéric Blanche & Anders Holt & Thomas Scheike, 2023. "On logistic regression with right censored data, with or without competing risks, and its use for estimating treatment effects," Lifetime Data Analysis: An International Journal Devoted to Statistical Methods and Applications for Time-to-Event Data, Springer, vol. 29(2), pages 441-482, April.
    5. Matthias Schmid & Thomas Hielscher & Thomas Augustin & Olaf Gefeller, 2011. "A Robust Alternative to the Schemper–Henderson Estimator of Prediction Error," Biometrics, The International Biometric Society, vol. 67(2), pages 524-535, June.
    6. Dendramis, Y. & Tzavalis, E. & Varthalitis, P. & Athanasiou, E., 2020. "Predicting default risk under asymmetric binary link functions," International Journal of Forecasting, Elsevier, vol. 36(3), pages 1039-1056.
    7. Ruosha Li & Limin Peng, 2017. "Assessing quantile prediction with censored quantile regression models," Biometrics, The International Biometric Society, vol. 73(2), pages 517-528, June.
    8. A. Gregory DiRienzo, 2009. "Flexible Regression Model Selection for Survival Probabilities: With Application to AIDS," Biometrics, The International Biometric Society, vol. 65(4), pages 1194-1202, December.
    9. Hajime Uno & Tianxi Cai & Lu Tian & L. J. Wei, 2011. "Graphical Procedures for Evaluating Overall and Subject-Specific Incremental Values from New Predictors with Censored Event Time Data," Biometrics, The International Biometric Society, vol. 67(4), pages 1389-1396, December.
    10. Meijer Rosa J. & Krebs Thijmen J.P. & Goeman Jelle J., 2015. "A region-based multiple testing method for hypotheses ordered in space or time," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 14(1), pages 1-19, February.
    11. Tianxi Cai & Thomas A Gerds & Yingye Zheng & Jinbo Chen, 2011. "Robust Prediction of t-Year Survival with Data from Multiple Studies," Biometrics, The International Biometric Society, vol. 67(2), pages 436-444, June.
    12. Signe M Jensen & Hanne Hauger & Christian Ritz, 2018. "Mediation analysis for logistic regression with interactions: Application of a surrogate marker in ophthalmology," PLOS ONE, Public Library of Science, vol. 13(2), pages 1-7, February.
    13. Klingenberg, Bernhard & Satopää, Ville, 2013. "Simultaneous confidence intervals for comparing margins of multivariate binary data," Computational Statistics & Data Analysis, Elsevier, vol. 64(C), pages 87-98.
    14. Shu Jiang & Jiguo Cao & Bernard Rosner & Graham A. Colditz, 2023. "Supervised two‐dimensional functional principal component analysis with time‐to‐event outcomes and mammogram imaging data," Biometrics, The International Biometric Society, vol. 79(2), pages 1359-1369, June.
    15. Christian Ritz & Rikke Pilmann Laursen & Camilla Trab Damsgaard, 2017. "Simultaneous inference for multilevel linear mixed models—with an application to a large-scale school meal study," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 66(2), pages 295-311, February.
    16. Rebecca Payne & Ming Yang & Yingye Zheng & Majken K. Jensen & Tianxi Cai, 2016. "Robust risk prediction with biomarkers under two‐phase stratified cohort design," Biometrics, The International Biometric Society, vol. 72(4), pages 1037-1045, December.
    17. Schmid, Matthias & Tutz, Gerhard & Welchowski, Thomas, 2018. "Discrimination measures for discrete time-to-event predictions," Econometrics and Statistics, Elsevier, vol. 7(C), pages 153-164.
    18. Holger Dette & Laura Hoyden & Sonja Kuhnt & Kirsten Schorning, 2017. "Optimal designs for thermal spraying," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 66(1), pages 53-72, January.
    19. Patrizia Pochetti & Danila Azzolina & Beatrice Ragnoli & Paolo Amedeo Tillio & Vincenzo Cantaluppi & Mario Malerba, 2020. "Interrelationship among Obstructive Sleep Apnea, Renal Function and Survival: A Cohort Study," IJERPH, MDPI, vol. 17(14), pages 1-12, July.
    20. Frank Schaarschmidt & Christian Ritz & Ludwig A. Hothorn, 2022. "The Tukey trend test: Multiplicity adjustment using multiple marginal models," Biometrics, The International Biometric Society, vol. 78(2), pages 789-797, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:biomet:v:78:y:2022:i:1:p:352-363. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0006-341X .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.