Efficient confidence intervals for the difference of two Bernoulli distributions’ success parameters

We study properties of confidence intervals (CIs) for the difference of two Bernoulli distributions’ success parameters. The CIs under investigation range from the classical fixed-sample-size CI to sequential versions, possibly incorporating batching. For each CI method, we examine the attained coverage, as well as the trade-offs between the number of observations and stages required to obtain a desired CI width. We consider cases in which the two populations are completely independent, and we provide analytical and simulation results to measure the performance of the different methods. For the multi-stage methods, we find that a simple observation allocation rule based on comparing the sample standard deviations of the two populations is more efficient than taking equal sample sizes from both. We also show that the use of a moderate level of batching saves stages at only modest costs in sample size and coverage.