A Double Application of the Benjamini-Hochberg Procedure for Testing Batched Hypotheses

The Benjamini-Hochberg procedure (BH) controls the false discovery rate (FDR), and on a large dataset optimizes signal discovery subject to this control. However it applies a common p-value rejection threshold that precludes it from taking advantage of index information of the null hypotheses, making it suboptimal for detecting clustered signals. We propose a double application of the BH procedure on two-level hierarchical and related datasets, the first application to identify p-value batches, and a second application on each identified batch for null hypotheses rejections. We propose a mixture model on two tiers to model signal clustering, and show that on this model, the double application reduces FDR and maintains the power of BH. We show that the doubly applied BH satisfies an average FDR control. Benjamini and Bogomolov (J R Stat Soc Ser B 76:297–318, 2014) considered a more general class of procedures and error criterions, and showed average FDR control under dependency assumptions different from ours. Their proof is also technically different. We end the paper with a description of Yekutieli’s (J Am Stat Assoc 103:309–316, 2008) procedure on hierarchical datasets, and a proposed hybrid of the double BH procedure and Yekutieli’s procedure that combines the strengths of both.