The Romano–Wolf multiple-hypothesis correction in Stata

When considering multiple-hypothesis tests simultaneously, standard statistical techniques will lead to overrejection of null hypotheses unless the multi- plicity of the testing framework is explicitly considered. In this article, we discuss the Romano–Wolf multiple-hypothesis correction and document its implementa- tion in Stata. The Romano–Wolf correction (asymptotically) controls the fami- lywise error rate, that is, the probability of rejecting at least one true null hy- pothesis among a family of hypotheses under test. This correction is considerably more powerful than earlier multiple-testing procedures, such as the Bonferroni and Holm corrections, given that it takes into account the dependence structure of the test statistics by resampling from the original data. We describe a command, rwolf, that implements this correction and provide several examples based on a wide range of models. We document and discuss the performance gains from us- ing rwolf over other multiple-testing procedures that control the familywise error rate.