IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v78y2014icp69-81.html

Three-group ROC analysis: A nonparametric predictive approach

Author

Listed:
  • Coolen-Maturi, Tahani
  • Elkhafifi, Faiza F.
  • Coolen, Frank P.A.

Abstract

Measuring the accuracy of diagnostic tests is crucial in many application areas, in particular medicine and health care. The receiver operating characteristic (ROC) surface is a useful tool to assess the ability of a diagnostic test to discriminate among three ordered classes or groups. Nonparametric predictive inference (NPI) is a frequentist statistical method that is explicitly aimed at using few modelling assumptions in addition to data, enabled through the use of lower and upper probabilities to quantify uncertainty. It focuses exclusively on a future observation, which may be particularly relevant if one considers decisions about a diagnostic test to be applied to a future patient. The NPI approach to three-group ROC analysis is presented, including results on the volumes under the ROC surfaces and choice of decision threshold for the diagnosis.

Suggested Citation

  • Coolen-Maturi, Tahani & Elkhafifi, Faiza F. & Coolen, Frank P.A., 2014. "Three-group ROC analysis: A nonparametric predictive approach," Computational Statistics & Data Analysis, Elsevier, vol. 78(C), pages 69-81.
    • Handle: RePEc:eee:csdana:v:78:y:2014:i:c:p:69-81
    DOI: 10.1016/j.csda.2014.04.005
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947314001017
    Download Restriction: Full text for ScienceDirect subscribers only.
    File URL: https://libkey.io/10.1016/j.csda.2014.04.005?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Chen, Weijie & Yousef, Waleed A. & Gallas, Brandon D. & Hsu, Elizabeth R. & Lababidi, Samir & Tang, Rong & Pennello, Gene A. & Symmans, W. Fraser & Pusztai, Lajos, 2012. "Uncertainty estimation with a finite dataset in the assessment of classification models," Computational Statistics & Data Analysis, Elsevier, vol. 56(5), pages 1016-1027.
    2. Zhang, Biao, 2006. "A semiparametric hypothesis testing procedure for the ROC curve area under a density ratio model," Computational Statistics & Data Analysis, Elsevier, vol. 50(7), pages 1855-1876, April.
    3. Stylianos Z. Xanthopoulos & Christos T. Nakas, 2007. "A generalized ROC approach for the validation of credit rating systems and scorecards," Journal of Risk Finance, Emerald Group Publishing, vol. 8(5), pages 481-488, November.
    4. Waegeman, Willem & De Baets, Bernard & Boullart, Luc, 2008. "On the scalability of ordered multi-class ROC analysis," Computational Statistics & Data Analysis, Elsevier, vol. 52(7), pages 3371-3388, March.
    5. Lai, Chin-Ying & Tian, Lili & Schisterman, Enrique F., 2012. "Exact confidence interval estimation for the Youden index and its corresponding optimal cut-point," Computational Statistics & Data Analysis, Elsevier, vol. 56(5), pages 1103-1114.
    6. J. F. Lawless & Marc Fredette, 2005. "Frequentist prediction intervals and predictive distributions," Biometrika, Biometrika Trust, vol. 92(3), pages 529-542, September.
    7. Rodríguez-Álvarez, María Xosé & Roca-Pardiñas, Javier & Cadarso-Suárez, Carmen, 2011. "A new flexible direct ROC regression model: Application to the detection of cardiovascular risk factors by anthropometric measures," Computational Statistics & Data Analysis, Elsevier, vol. 55(12), pages 3257-3270, December.
    8. Adler, Werner & Lausen, Berthold, 2009. "Bootstrap estimated true and false positive rates and ROC curve," Computational Statistics & Data Analysis, Elsevier, vol. 53(3), pages 718-729, January.
    9. Rodríguez-Álvarez, María Xosé & Tahoces, Pablo G. & Cadarso-Suárez, Carmen & Lado, María José, 2011. "Comparative study of ROC regression techniques--Applications for the computer-aided diagnostic system in breast cancer detection," Computational Statistics & Data Analysis, Elsevier, vol. 55(1), pages 888-902, January.
    10. Christos T. Nakas & Todd A. Alonzo, 2007. "ROC Graphs for Assessing the Ability of a Diagnostic Marker to Detect Three Disease Classes with an Umbrella Ordering," Biometrics, The International Biometric Society, vol. 63(2), pages 603-609, June.
    11. Lopez-de-Ullibarri, Ignacio & Cao, Ricardo & Cadarso-Suarez, Carmen & Lado, Maria J., 2008. "Nonparametric estimation of conditional ROC curves: Application to discrimination tasks in computerized detection of early breast cancer," Computational Statistics & Data Analysis, Elsevier, vol. 52(5), pages 2623-2631, January.
    12. Kang, Le & Tian, Lili, 2013. "Estimation of the volume under the ROC surface with three ordinal diagnostic categories," Computational Statistics & Data Analysis, Elsevier, vol. 62(C), pages 39-51.
    13. Yousef, Waleed A. & Kundu, Subrata & Wagner, Robert F., 2009. "Nonparametric estimation of the threshold at an operating point on the ROC curve," Computational Statistics & Data Analysis, Elsevier, vol. 53(12), pages 4370-4383, October.
    14. Wan, Shuwen & Zhang, Biao, 2008. "Comparing correlated ROC curves for continuous diagnostic tests under density ratio models," Computational Statistics & Data Analysis, Elsevier, vol. 53(1), pages 233-245, September.
    15. Airola, Antti & Pahikkala, Tapio & Waegeman, Willem & De Baets, Bernard & Salakoski, Tapio, 2011. "An experimental comparison of cross-validation techniques for estimating the area under the ROC curve," Computational Statistics & Data Analysis, Elsevier, vol. 55(4), pages 1828-1844, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pardo-Fernandez, Juan Carlos & Rodriguez-alvarez, Maria Xose & Van Keilegom, Ingrid, 2013. "A review on ROC curves in the presence of covariates," LIDAM Discussion Papers ISBA 2013050, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    2. Rodríguez-Álvarez, María Xosé & Roca-Pardiñas, Javier & Cadarso-Suárez, Carmen, 2011. "A new flexible direct ROC regression model: Application to the detection of cardiovascular risk factors by anthropometric measures," Computational Statistics & Data Analysis, Elsevier, vol. 55(12), pages 3257-3270, December.
    3. Chen, Zhelun & O’Neill, Zheng & Wen, Jin & Pradhan, Ojas & Yang, Tao & Lu, Xing & Lin, Guanjing & Miyata, Shohei & Lee, Seungjae & Shen, Chou & Chiosa, Roberto & Piscitelli, Marco Savino & Capozzoli, , 2023. "A review of data-driven fault detection and diagnostics for building HVAC systems," Applied Energy, Elsevier, vol. 339(C).
    4. Poon, Wai-Yin & Qiu, Shi-Fang & Tang, Man-Lai, 2015. "Confidence interval construction for the Youden index based on partially validated series," Computational Statistics & Data Analysis, Elsevier, vol. 84(C), pages 116-134.
    5. Acharki, Naoufal & Bertoncello, Antoine & Garnier, Josselin, 2023. "Robust prediction interval estimation for Gaussian processes by cross-validation method," Computational Statistics & Data Analysis, Elsevier, vol. 178(C).
    6. Qiu, Zhiping & Peng, Limin & Manatunga, Amita & Guo, Ying, 2019. "A smooth nonparametric approach to determining cut-points of a continuous scale," Computational Statistics & Data Analysis, Elsevier, vol. 134(C), pages 186-210.
    7. Khanh To Duc & Monica Chiogna & Gianfranco Adimari, 2019. "Estimation of the volume under the ROC surface in presence of nonignorable verification bias," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 28(4), pages 695-722, December.
    8. Pablo Martínez-Camblor & Susana Díaz-Coto, 2024. "Reducing the overfitting in the gROC curve estimation," Computational Statistics, Springer, vol. 39(2), pages 1005-1022, April.
    9. Doyo Gragn Enki & Angela Noufaily & Paddy Farrington & Paul Garthwaite & Nick Andrews & Andre Charlett, 2017. "Taylor's power law and the statistical modelling of infectious disease surveillance data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(1), pages 45-72, January.
    10. Seyed Poorya Mirfallah Lialestani & David Parcerisa & Mahjoub Himi & Abbas Abbaszadeh Shahri, 2022. "Generating 3D Geothermal Maps in Catalonia, Spain Using a Hybrid Adaptive Multitask Deep Learning Procedure," Energies, MDPI, vol. 15(13), pages 1-16, June.
    11. Jorge Navarro & Francesco Buono, 2023. "Predicting future failure times by using quantile regression," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 86(5), pages 543-576, July.
    12. Paolo Vidoni, 2009. "Improved Prediction Intervals and Distribution Functions," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 36(4), pages 735-748, December.
    13. P Coolen-Schrijner & F P A Coolen & I M MacPhee, 2008. "Nonparametric predictive inference for system reliability with redundancy allocation," Journal of Risk and Reliability, , vol. 222(4), pages 463-476, December.
    14. Edler, Lutz & Lee, Jae Won & Mittlböck, Martina & Niland, Joyce & Victor, Norbert, 2009. "Computational statistics within clinical research," Computational Statistics & Data Analysis, Elsevier, vol. 53(3), pages 583-585, January.
    15. Grzegorz Wyłupek, 2016. "An Automatic Test for the Umbrella Alternatives," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 43(4), pages 1103-1123, December.
    16. repec:plo:pone00:0158843 is not listed on IDEAS
    17. Christos Nakas, 2025. "Robustness of the Trinormal ROC Surface Model: Formal Assessment via Goodness-of-Fit Testing," Stats, MDPI, vol. 8(4), pages 1-15, October.
    18. Yuanyuan Shen & Katherine P. Liao & Tianxi Cai, 2015. "Sparse kernel machine regression for ordinal outcomes," Biometrics, The International Biometric Society, vol. 71(1), pages 63-70, March.
    19. Rota, Matteo & Antolini, Laura, 2014. "Finding the optimal cut-point for Gaussian and Gamma distributed biomarkers," Computational Statistics & Data Analysis, Elsevier, vol. 69(C), pages 1-14.
    20. Matteo Borrotti, 2024. "Quantifying Uncertainty with Conformal Prediction for Heating and Cooling Load Forecasting in Building Performance Simulation," Energies, MDPI, vol. 17(17), pages 1-13, August.
    21. Wang, Suohong & Zhang, Biao, 2014. "Semiparametric empirical likelihood confidence intervals for AUC under a density ratio model," Computational Statistics & Data Analysis, Elsevier, vol. 70(C), pages 101-115.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:78:y:2014:i:c:p:69-81. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.