IDEAS home Printed from https://ideas.repec.org/a/tsj/stataj/v9y2009i1p1-16.html
   My bibliography  Save this article

Estimation and comparison of receiver operating characteristic curves

Author

Listed:
  • Margaret S. Pepe

    (Fred Hutchinson Cancer Research Center)

  • Gary Longton

    (Fred Hutchinson Cancer Research Center)

  • Holly Janes

    (Fred Hutchinson Cancer Research Center)

Abstract

The receiver operating characteristic (ROC) curve displays the capacity of a marker or diagnostic test to discriminate between two groups of subjects, cases versus controls. We present a comprehensive suite of Stata commands for performing ROC analysis. Nonparametric, semiparametric, and parametric estimators are calculated. Comparisons between curves are based on the area or partial area under the ROC curve. Alternatively, pointwise comparisons between ROC curves or inverse ROC curves can be made. We describe options to adjust these analyses for covariates and to perform ROC regression in a companion article. We use a unified framework by representing the ROC curve as the distribution of the marker in cases where we have standardized it to the control reference distribution. Copyright 2009 by StataCorp LP.

Suggested Citation

  • Margaret S. Pepe & Gary Longton & Holly Janes, 2009. "Estimation and comparison of receiver operating characteristic curves," Stata Journal, StataCorp LP, vol. 9(1), pages 1-16, March.
    • Handle: RePEc:tsj:stataj:v:9:y:2009:i:1:p:1-16
    as

    Download full text from publisher

    File URL: http://www.stata-journal.com/article.html?article=st0154
    Download Restriction: no
    File URL: http://www.stata-journal.com/software/sj9-1/st0154/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Lori E. Dodd & Margaret S. Pepe, 2003. "Partial AUC Estimation and Regression," Biometrics, The International Biometric Society, vol. 59(3), pages 614-623, September.
    2. Margaret Sullivan Pepe & Tianxi Cai, 2004. "The Analysis of Placement Values for Evaluating Discriminatory Measures," Biometrics, The International Biometric Society, vol. 60(2), pages 528-535, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Drehmann, Mathias & Juselius, Mikael, 2014. "Evaluating early warning indicators of banking crises: Satisfying policy requirements," International Journal of Forecasting, Elsevier, vol. 30(3), pages 759-780.
    2. Bespalova, Olga, 2018. "Forecast Evaluation in Macroeconomics and International Finance. Ph.D. thesis, George Washington University, Washington, DC, USA," MPRA Paper 117706, University Library of Munich, Germany.
    3. Nancy Puttkammer & Steven Zeliadt & Jean Gabriel Balan & Janet Baseman & Rodney Destiné & Jean Wysler Domerçant & Garilus France & Nathaelf Hyppolite & Valérie Pelletier & Nernst Atwood Raphael & Kenn, 2014. "Development of an Electronic Medical Record Based Alert for Risk of HIV Treatment Failure in a Low-Resource Setting," PLOS ONE, Public Library of Science, vol. 9(11), pages 1-12, November.
    4. Aastveit, Knut Are & Anundsen, André K. & Herstad, Eyo I., 2019. "Residential investment and recession predictability," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1790-1799.
    5. John Muschelli, 2020. "ROC and AUC with a Binary Predictor: a Potentially Misleading Metric," Journal of Classification, Springer;The Classification Society, vol. 37(3), pages 696-708, October.
    6. André K. Anundsen & Karsten Gerdrup & Frank Hansen & Kasper Kragh‐Sørensen, 2016. "Bubbles and Crises: The Role of House Prices and Credit," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(7), pages 1291-1311, November.
    7. Carsten Detken & Olaf Weeken & Lucia Alessi & Diana Bonfim & Miguel M. Boucinha & Christian Castro & Sebastian Frontczak & Gaston Giordana & Julia Giese & Nadya Jahn & Jan Kakes & Benjamin Klaus & Jan, 2014. "Operationalising the countercyclical capital buffer: indicator selection, threshold identification and calibration options," ESRB Occasional Paper Series 05, European Systemic Risk Board.
    8. Jung Wha Chung & Beom Hee Kim & Chung Seop Lee & Gi Hyun Kim & Hyung Rae Sohn & Bo Young Min & Joon Chang Song & Hyun Kyung Park & Eun Sun Jang & Hyuk Yoon & Jaihwan Kim & Cheol Min Shin & Young Soo P, 2016. "Optimizing Surveillance Performance of Alpha-Fetoprotein by Selection of Proper Target Population in Chronic Hepatitis B," PLOS ONE, Public Library of Science, vol. 11(12), pages 1-13, December.
    9. Holly Janes & Gary Longton & Margaret S. Pepe, 2009. "Accommodating covariates in receiver operating characteristic analysis," Stata Journal, StataCorp LP, vol. 9(1), pages 17-39, March.
    10. Oke Gerke & Antonia Zapf, 2022. "Convergence Behavior of Optimal Cut-Off Points Derived from Receiver Operating Characteristics Curve Analysis: A Simulation Study," Mathematics, MDPI, vol. 10(22), pages 1-14, November.
    11. Jesús F. Lampón & Pablo Cabanelas-Lorenzo & Santiago Lago-Peñas, 2013. "Why firms relocate their production overseas? The answer lies inside: corporate, logistic and technological determinants," Working Papers 2013/3, Institut d'Economia de Barcelona (IEB).
    12. Mund, Carolin & Neuhäusler, Peter, 2015. "Towards an early-stage identification of emerging topics in science—The usability of bibliometric characteristics," Journal of Informetrics, Elsevier, vol. 9(4), pages 1018-1033.
    13. Marcin Łupiński, 2019. "Wskaźniki wczesnego ostrzegania przed niestabilnością finansową polskiego sektora bankowego," Collegium of Economic Analysis Annals, Warsaw School of Economics, Collegium of Economic Analysis, issue 55, pages 99-113.
    14. Rocío Aznar-Gimeno & Luis M. Esteban & Gorka Labata-Lezaun & Rafael del-Hoyo-Alonso & David Abadia-Gallego & J. Ramón Paño-Pardo & M. José Esquillor-Rodrigo & Ángel Lanas & M. Trinidad Serrano, 2021. "A Clinical Decision Web to Predict ICU Admission or Death for Patients Hospitalised with COVID-19 Using Machine Learning Algorithms," IJERPH, MDPI, vol. 18(16), pages 1-20, August.
    15. Jesús F. Lampón & Pablo Cabanelas-Lorenzo & Santiago Lago-Peñas, 2013. "Why firms relocate their production overseas? The answer lies inside: corporate, logistic and technological determinants," Working Papers 2013/3, Institut d'Economia de Barcelona (IEB).
    16. Detken, Carsten & Weeken, Olaf & Alessi, Lucia & Bonfim, Diana & Boucinha, Miguel & Castro, Christian & Frontczak, Sebastian & Giordana, Gaston & Giese, Julia & Wildmann, Nadya & Kakes, Jan & Klaus, B, 2014. "Operationalising the countercyclical capital buffer: indicator selection, threshold identification and calibration options," ESRB Occasional Paper Series 5, European Systemic Risk Board.
    17. Elizabeth Gutierrez & Jake Krupa & Miguel Minutti-Meza & Maria Vulcheva, 2020. "Do going concern opinions provide incremental information to predict corporate defaults?," Review of Accounting Studies, Springer, vol. 25(4), pages 1344-1381, December.
    18. Geršl, Adam & Jašová, Martina, 2018. "Credit-based early warning indicators of banking crises in emerging markets," Economic Systems, Elsevier, vol. 42(1), pages 18-31.
    19. Eva V Applebaum & Dominic Breton & Zhuo Wei Feng & An-Tchi Ta & Kayley Walsh & Kathleen Chassé & Shawn M Robbins, 2017. "Modified 30-second Sit to Stand test predicts falls in a cohort of institutionalized older veterans," PLOS ONE, Public Library of Science, vol. 12(5), pages 1-13, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Soutik Ghosal & Zhen Chen, 2022. "Discriminatory Capacity of Prenatal Ultrasound Measures for Large-for-Gestational-Age Birth: A Bayesian Approach to ROC Analysis Using Placement Values," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 14(1), pages 1-22, April.
    2. Holly Janes & Gary Longton & Margaret S. Pepe, 2009. "Accommodating covariates in receiver operating characteristic analysis," Stata Journal, StataCorp LP, vol. 9(1), pages 17-39, March.
    3. Tianxi Cai & Yingye Zheng, 2007. "Model Checking for ROC Regression Analysis," Biometrics, The International Biometric Society, vol. 63(1), pages 152-163, March.
    4. Y. Huang & M. S. Pepe, 2009. "A Parametric ROC Model-Based Approach for Evaluating the Predictiveness of Continuous Markers in Case–Control Studies," Biometrics, The International Biometric Society, vol. 65(4), pages 1133-1144, December.
    5. Peterson, A. Townsend & Papeş, Monica & Soberón, Jorge, 2008. "Rethinking receiver operating characteristic analysis applications in ecological niche modeling," Ecological Modelling, Elsevier, vol. 213(1), pages 63-72.
    6. Margaret Sullivan Pepe & Tianxi Cai, 2004. "The Analysis of Placement Values for Evaluating Discriminatory Measures," Biometrics, The International Biometric Society, vol. 60(2), pages 528-535, June.
    7. Elisa–María Molanes-López & Ricardo Cao, 2008. "Relative density estimation for left truncated and right censored data," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 20(8), pages 693-720.
    8. Man-Jen Hsu & Huey-Miin Hsueh, 2013. "The linear combinations of biomarkers which maximize the partial area under the ROC curves," Computational Statistics, Springer, vol. 28(2), pages 647-666, April.
    9. Chen, Xiwei & Vexler, Albert & Markatou, Marianthi, 2015. "Empirical likelihood ratio confidence interval estimation of best linear combinations of biomarkers," Computational Statistics & Data Analysis, Elsevier, vol. 82(C), pages 186-198.
    10. Rodríguez-Álvarez, María Xosé & Roca-Pardiñas, Javier & Cadarso-Suárez, Carmen, 2011. "A new flexible direct ROC regression model: Application to the detection of cardiovascular risk factors by anthropometric measures," Computational Statistics & Data Analysis, Elsevier, vol. 55(12), pages 3257-3270, December.
    11. Gigliarano, Chiara & Figini, Silvia & Muliere, Pietro, 2014. "Making classifier performance comparisons when ROC curves intersect," Computational Statistics & Data Analysis, Elsevier, vol. 77(C), pages 300-312.
    12. Yu, Wenbao & Park, Taesung, 2015. "Two simple algorithms on linear combination of multiple biomarkers to maximize partial area under the ROC curve," Computational Statistics & Data Analysis, Elsevier, vol. 88(C), pages 15-27.
    13. Yousef, Waleed A., 2013. "Assessing classifiers in terms of the partial area under the ROC curve," Computational Statistics & Data Analysis, Elsevier, vol. 64(C), pages 51-70.
    14. Pardo-Fernandez, Juan Carlos & Rodriguez-alvarez, Maria Xose & Van Keilegom, Ingrid, 2013. "A review on ROC curves in the presence of covariates," LIDAM Discussion Papers ISBA 2013050, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    15. Mei-Cheng Wang & Shanshan Li, 2012. "Bivariate Marker Measurements and ROC Analysis," Biometrics, The International Biometric Society, vol. 68(4), pages 1207-1218, December.
    16. Eunhee Kim & Zheng Zhang & Youdan Wang & Donglin Zeng, 2014. "Power calculation for comparing diagnostic accuracies in a multi-reader, multi-test design," Biometrics, The International Biometric Society, vol. 70(4), pages 1033-1041, December.
    17. Merve Basol & Dincer Goksuluk & Ergun Karaagaoglu, 2023. "Comparing the diagnostic performance of methods used in a full-factorial design multi-reader multi-case studies," Computational Statistics, Springer, vol. 38(3), pages 1537-1553, September.
    18. Jialiang Li & Jason P. Fine, 2010. "Weighted area under the receiver operating characteristic curve and its application to gene selection," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 59(4), pages 673-692, August.
    19. Wang, Suohong & Zhang, Biao, 2014. "Semiparametric empirical likelihood confidence intervals for AUC under a density ratio model," Computational Statistics & Data Analysis, Elsevier, vol. 70(C), pages 101-115.
    20. Eleni Verykouki & Christos T. Nakas, 2023. "Adaptations on the Use of p -Values for Statistical Inference: An Interpretation of Messages from Recent Public Discussions," Stats, MDPI, vol. 6(2), pages 1-13, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tsj:stataj:v:9:y:2009:i:1:p:1-16. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christopher F. Baum or Lisa Gilmore (email available below). General contact details of provider: http://www.stata-journal.com/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.