cvauroc: Command to compute cross-validated area under the curve for ROC analysis after predictive modeling for binary outcomes

Receiver operating characteristic (ROC) analysis is used for comparing predictive models in both model selection and model evaluation. ROC analysis is often applied in clinical medicine and social science to assess the tradeoff be- tween model sensitivity and specificity. After fitting a binary logistic or probit regression model with a set of independent variables, the predictive performance of this set of variables can be assessed by the area under the curve (AUC) from an ROC curve. An important aspect of predictive modeling (regardless of model type) is the ability of a model to generalize to new cases. Evaluating the predic- tive performance (AUC) of a set of independent variables using all cases from the original analysis sample often results in an overly optimistic estimate of predictive performance. One can use K-fold cross-validation to generate a more realistic esti- mate of predictive performance in situations with a small number of observations. AUC is estimated iteratively for k samples (the “test” samples) that are indepen- dent of the sample used to predict the dependent variable (the “training” sample). cvauroc implements k-fold cross-validation for the AUC for a binary outcome af- ter fitting a logit or probit regression model, averaging the AUCs corresponding to each fold, and bootstrapping the cross-validated AUC to obtain statistical in- ference and 95% confidence intervals. Furthermore, cvauroc optionally provides the cross-validated fitted probabilities for the dependent variable or outcome, con- tained in a new variable named fit; the sensitivity and specificity for each of the levels of the predicted outcome, contained in two new variables named sen and spe; and the plot of the mean cross-validated AUC and k-fold ROC curves.