Single criterion for rating confidence interval procedures: a generalization to stochastic stopping rules

In this paper, we propose a generalization of the VAMP1RE criterion, which was used to rank and rate fixed sample size confidence interval procedures (CIPs), to be able to rank CIPs that use stochastic stopping rules. The key aspects of VAMP1RE remain unchanged in the generalized version. Specifically, we intend for it to serve as a single criterion, which measures the mean squared error of the difference between the coverage values of the CIP of interest and an ideal CIP. Moreover, we advocate that the quality of an interval should be evaluated based on its similarity to the interval produced by the ideal CIP, rather than solely on its width. We estimate the criterion using Monte Carlo simulation with Schruben’s coverage function. The generalization involves two modifications. The first modification replaces the arbitrary selection of nominal coverage values with averaging over a specified distribution of such values. The second modification replaces the use of a fixed sample size with a stopping condition that depends on the CIP of interest. We present experimental results that show how the generalized VAMP1RE criterion can be used to rate CIPs with stochastic stopping rules by providing useful numerical values.