On the estimation of inverse-probability-of-censoring weights for the evaluation of survival prediction error

Inverse probability weighting (IPW) is a popular method for making inferences regarding unobserved or unobservable data of a target population based on observed data. This paper considers IPW applied to right-censored time-to-event data. We investigate the behavior of the inverse-probability-of-censoring weighted (IPCW) Brier score, which is frequently used to assess the predictive performance of time-to-event models. A key requirement of the IPCW Brier score is the estimation of the censoring distribution, which is needed to compute the weights. The established paradigm of splitting a dataset into a training and a test set for model fitting and evaluation raises the question which of these datasets to use in order to fit the censoring model. There seems to be considerable disagreement between authors with regards to this issue, and no standard has been established so far. To shed light on this important question, we conducted a comprehensive experimental study exploring various data scenarios and estimation schemes. We found that it is generally of little importance which dataset is used to model the censoring distribution. However, in some circumstances, such as in the case of a covariate-dependent censoring process, a small sample size, or when dealing with noisy data, it may be advisable to use the test set instead of the training set to model the censoring distribution. A detailed set of practical recommendations concludes our paper.