Surprisal intervals as an Extension of Confidence/Compatibility Intervals for S-Values: Abandonment of Statistical Significance in Hypothesis Testing

Despite decades of warnings, misuse and misinterpretation of statistical significance continue to be prevalent in science. The absence of simple and intuitive explanations of this concept often leads researchers to incorrect or misleading conclusions that can have severe consequences, especially in fields like medicine and public health. For this reason, some statisticians suggest adopting S-values (surprisals) instead of P-values, as they relate the statistical relevance of an event to the number of consecutive heads when flipping an unbiased coin. Such a comparison with a phenomenon that we encounter in our daily lives makes S-values simpler to understand than P-values and reduces the likelihood of making overstatements. However, to the best of the author’s knowledge, there is currently no natural extension of confidence/compatibility intervals for statistical surprise. This asymmetry forces researchers to remain bound to the significance threshold α, which is necessary to define said intervals through the 100(1-α)% relationship. To address this issue, this paper introduces the concept of surprisal intervals, explaining their use through straightforward examples. Surprisal intervals enable the definitive abandonment of inferential evaluation of results and inherently prevent the dichotomous approach in hypothesis testing. Indeed, the proposed methodologies impose the assessment of the outcomes in terms of more and less surprising than some fixed values instead of significant and non-significant, thus making the notion of degree of surprise manifest and ineliminable. Based on the above considerations, the use of surprisal intervals is highly recommended in future scientific investigations.