Non-conformal Shortcut

In Chap. 8 we argued for complementing a prediction rule, i.e., a trained prediction algorithm, with a system for testing deviations from exchangeability. As soon as serious violations of exchangeability are detected, we start retraining the prediction algorithm or take other appropriate measures. In this short chapter we suggest a shortcut: we test directly the predictions output by the prediction rule, and a successful way of testing then automatically translates to improved (“protected”) predictions. In this way we apply methods developed in the first two chapters of this part outside conformal prediction. Our procedures of protected prediction can be used from the very beginning, or after detecting lack of exchangeability (but before retraining is complete). We consider separately the case of regression, in which we combine the probability integral transformation and betting martingales, and classification, in which we move even further from conformal prediction.