A more versatile sample size calculator

I present a new Stata command, simsam, that uses simulation to determine the sample size required to achieve given power for any method of analysis under any probability model that can be programmed in Stata (simsam assumes that code for generating a single dataset and analyzing it can be found in a separate program). Thus simsam extends Stata’s sampsi command. It is straightforward to estimate the power of a statistical analysis for a given sample size by simulation: you simply run the analysis repeatedly on simulated data and see how often the result is statistically significant. Determining the sample size that achieves given power is slightly harder, requiring power to be assessed at different sample sizes in order to find the one at which the target power is attained. simsam uses a novel iterative algorithm that is more efficient than stepping consecutively through every possible sample size. The user specifies the precision of the final estimate of power, but initially the algorithm uses less precision in order to make more rapid progress. The algorithm aims for the smallest sample size (or the smallest multiple of 5, or 10, or other user-specified increment) such that the power, estimated to within the specified precision, exceeds the target power. The power is reported with a 99% confidence interval, and the final sample size obtained is reliable in the sense that if the simsam command is repeated (by the same user or by a reviewer checking the calculation), it will, nearly every time, give a sample size no more than one increment away.